Compare commits

..

14 Commits

Author SHA1 Message Date
grandwizard28
8bea74f077 fix(audit): remove debug assertion from integration test 2026-04-03 20:00:36 +05:30
grandwizard28
9726371b79 fix(audit): revert sb.As in getAuditKeys, fix fixture column_names
Revert getAuditKeys to use raw SQL strings instead of sb.As() which
incorrectly treated string literals as column references. Add explicit
column_names to all ClickHouse insert calls in the audit fixture.
2026-04-03 18:02:01 +05:30
grandwizard28
c71b93f36d test(audit): add integration tests for audit log querying
Cover the documented query patterns: list all events, filter by
principal ID, filter by outcome, filter by resource name+ID,
filter by principal type, scalar count for alerting, and
isolation test ensuring audit data doesn't leak into regular logs.
2026-04-03 16:44:30 +05:30
grandwizard28
8d8406d7f0 feat(audit): wire audit statement builder into querier
Add auditStmtBuilder to querier struct and route LogAggregation
queries with source=audit to it in all three dispatch locations
(main query, live tail, shiftedQuery). Create and wire the full
audit query stack in signozquerier provider.
2026-04-03 16:33:44 +05:30
grandwizard28
e84c75d41f fix(audit): rename tag_attributes_v2 to tag_attributes
Migration uses tag_attributes/distributed_tag_attributes (no _v2
suffix). Rename constants and update all references including the
integration test fixture.
2026-04-03 16:12:31 +05:30
grandwizard28
a469fbab14 feat(audit): add integration test fixture for audit logs
AuditLog fixture inserts into all 5 signoz_audit tables matching
the schema migration DDL: distributed_logs (no resources_string,
has event_name), distributed_logs_resource, distributed_tag_attributes_v2,
distributed_logs_attribute_keys, distributed_logs_resource_keys.
2026-04-03 15:34:20 +05:30
grandwizard28
3835a0f4b0 fix(audit): align table names with schema migration
Migration uses logs/distributed_logs (not logs_v2/distributed_logs_v2).
Rename LogsV2TableName to LogsTableName and LogsV2LocalTableName to
LogsLocalTableName to match the actual signoz_audit DDL.
2026-04-03 15:27:41 +05:30
grandwizard28
9f6cc9c4eb style(audit): move column map to const.go, use sqlbuilder.As in metadata
Move logsV2Columns from field_mapper.go to const.go to colocate all
column definitions. Switch getAuditKeys() to use sb.As() instead of
raw string formatting. Fix FieldContext alignment.
2026-04-03 00:38:58 +05:30
grandwizard28
d1ae4b6173 refactor(audit): inline field key map into test file
Remove test_data.go and inline the audit field key map directly
into statement_builder_test.go with a compact helper function.
2026-04-03 00:20:07 +05:30
grandwizard28
1e3dc8de4f test(audit): add statement builder tests
Cover all three request types (list, time series, scalar) with
audit-specific query patterns: materialized column filters, AND/OR
conditions, limit CTEs, and group-by expressions.
2026-04-03 00:16:31 +05:30
grandwizard28
a24b75d071 feat(audit): add audit field value autocomplete support
Wire distributed_tag_attributes_v2 for signoz_audit into the
metadata store. Add getAuditFieldValues() and route SignalLogs +
SourceAudit to it in GetFieldValues().
2026-04-02 23:12:21 +05:30
grandwizard28
5db81ba7f6 fix(audit): align field mapper with actual audit DDL schema
Remove resources_string (not in audit table DDL).
Add event_name as intrinsic column.
Resource context resolves only through the resource JSON column.
2026-04-02 23:00:26 +05:30
grandwizard28
77493f7c76 chore: address review comments
Comment out SourceAudit from Enum() until frontend is ready.
Use actual audit table constants in metadata test helpers.
2026-04-02 22:52:28 +05:30
grandwizard28
b1f7e5a3df feat(audit): add telemetry audit query infrastructure
Add pkg/telemetryaudit/ with tables, field mapper, condition builder,
and statement builder for querying audit logs from signoz_audit database.
Add SourceAudit to source enum and integrate audit key resolution
into the metadata store.
2026-04-02 16:41:58 +05:30
78 changed files with 2695 additions and 3487 deletions

View File

@@ -55,9 +55,6 @@ jobs:
sqlstore-provider:
- postgres
- sqlite
sqlite-mode:
- delete
- wal
clickhouse-version:
- 25.5.6
- 25.12.5
@@ -65,9 +62,6 @@ jobs:
- v0.142.0
postgres-version:
- 15
exclude:
- sqlstore-provider: postgres
sqlite-mode: wal
if: |
((github.event_name == 'pull_request' && ! github.event.pull_request.head.repo.fork && github.event.pull_request.user.login != 'dependabot[bot]' && ! contains(github.event.pull_request.labels.*.name, 'safe-to-test')) ||
(github.event_name == 'pull_request_target' && contains(github.event.pull_request.labels.*.name, 'safe-to-test'))) && contains(github.event.pull_request.labels.*.name, 'safe-to-integrate')
@@ -108,7 +102,6 @@ jobs:
--basetemp=./tmp/ \
src/${{matrix.src}} \
--sqlstore-provider ${{matrix.sqlstore-provider}} \
--sqlite-mode ${{matrix.sqlite-mode}} \
--postgres-version ${{matrix.postgres-version}} \
--clickhouse-version ${{matrix.clickhouse-version}} \
--schema-migrator-version ${{matrix.schema-migrator-version}}

View File

@@ -86,7 +86,7 @@ sqlstore:
# The path to the SQLite database file.
path: /var/lib/signoz/signoz.db
# The journal mode for the sqlite database. Supported values: delete, wal.
mode: wal
mode: delete
# The timeout for the sqlite database to wait for a lock.
busy_timeout: 10s
# The default transaction locking behavior. Supported values: deferred, immediate, exclusive.

View File

@@ -193,7 +193,6 @@ uv run pytest --basetemp=./tmp/ -vv --reuse src/passwordauthn/01_register.py::te
Tests can be configured using pytest options:
- `--sqlstore-provider` - Choose database provider (default: postgres)
- `--sqlite-mode` - SQLite journal mode: `delete` or `wal` (default: delete). Only relevant when `--sqlstore-provider=sqlite`.
- `--postgres-version` - PostgreSQL version (default: 15)
- `--clickhouse-version` - ClickHouse version (default: 25.5.6)
- `--zookeeper-version` - Zookeeper version (default: 3.7.1)
@@ -203,6 +202,7 @@ Example:
uv run pytest --basetemp=./tmp/ -vv --reuse --sqlstore-provider=postgres --postgres-version=14 src/auth/
```
## What should I remember?
- **Always use the `--reuse` flag** when setting up the environment to keep containers running
@@ -213,4 +213,3 @@ uv run pytest --basetemp=./tmp/ -vv --reuse --sqlstore-provider=postgres --postg
- **Use descriptive test names** that clearly indicate what is being tested
- **Leverage fixtures** for common setup and authentication
- **Test both success and failure scenarios** to ensure robust functionality
- **`--sqlite-mode=wal` does not work on macOS.** The integration test environment runs SigNoz inside a Linux container with the SQLite database file mounted from the macOS host. WAL mode requires shared memory between connections, and connections crossing the VM boundary (macOS host ↔ Linux container) cannot share the WAL index, resulting in `SQLITE_IOERR_SHORT_READ`. WAL mode is tested in CI on Linux only.

View File

@@ -68,8 +68,8 @@
"@signozhq/toggle-group": "0.0.1",
"@signozhq/tooltip": "0.0.2",
"@signozhq/ui": "0.0.5",
"@tanstack/react-table": "8.21.3",
"@tanstack/react-virtual": "3.13.22",
"@tanstack/react-table": "8.20.6",
"@tanstack/react-virtual": "3.11.2",
"@uiw/codemirror-theme-copilot": "4.23.11",
"@uiw/codemirror-theme-github": "4.24.1",
"@uiw/react-codemirror": "4.23.10",

View File

@@ -1,52 +0,0 @@
import { useEffect } from 'react';
// eslint-disable-next-line no-restricted-imports
import { useSelector } from 'react-redux';
import { refreshIntervalOptions } from 'container/TopNav/AutoRefreshV2/constants';
import { Time } from 'container/TopNav/DateTimeSelectionV2/types';
import { useGlobalTimeStore } from 'store/globalTime/globalTimeStore';
import { createCustomTimeRange } from 'store/globalTime/utils';
import { AppState } from 'store/reducers';
import { GlobalReducer } from 'types/reducer/globalTime';
/**
* Adapter component that syncs Redux global time state to Zustand store.
* This component should be rendered once at the app level.
*
* It reads from the Redux globalTime reducer and updates the Zustand store
* to provide a migration path from Redux to Zustand.
*/
export function GlobalTimeStoreAdapter(): null {
const globalTime = useSelector<AppState, GlobalReducer>(
(state) => state.globalTime,
);
const setSelectedTime = useGlobalTimeStore((s) => s.setSelectedTime);
useEffect(() => {
// Convert the selectedTime to the new format
// If it's 'custom', store the min/max times in the custom format
const selectedTime =
globalTime.selectedTime === 'custom'
? createCustomTimeRange(globalTime.minTime, globalTime.maxTime)
: (globalTime.selectedTime as Time);
// Find refresh interval from Redux state
const refreshOption = refreshIntervalOptions.find(
(option) => option.key === globalTime.selectedAutoRefreshInterval,
);
const refreshInterval =
!globalTime.isAutoRefreshDisabled && refreshOption ? refreshOption.value : 0;
setSelectedTime(selectedTime, refreshInterval);
}, [
globalTime.selectedTime,
globalTime.isAutoRefreshDisabled,
globalTime.selectedAutoRefreshInterval,
globalTime.minTime,
globalTime.maxTime,
setSelectedTime,
]);
return null;
}

View File

@@ -1,227 +0,0 @@
// eslint-disable-next-line no-restricted-imports
import { Provider } from 'react-redux';
import { act, render, renderHook } from '@testing-library/react';
import { DEFAULT_TIME_RANGE } from 'container/TopNav/DateTimeSelectionV2/constants';
import configureStore, { MockStoreEnhanced } from 'redux-mock-store';
import { useGlobalTimeStore } from 'store/globalTime/globalTimeStore';
import { createCustomTimeRange } from 'store/globalTime/utils';
import { AppState } from 'store/reducers';
import { GlobalReducer } from 'types/reducer/globalTime';
import { GlobalTimeStoreAdapter } from '../GlobalTimeStoreAdapter';
const mockStore = configureStore<Partial<AppState>>([]);
const randomTime = 1700000000000000000;
describe('GlobalTimeStoreAdapter', () => {
let store: MockStoreEnhanced<Partial<AppState>>;
const createGlobalTimeState = (
overrides: Partial<GlobalReducer> = {},
): GlobalReducer => ({
minTime: randomTime,
maxTime: randomTime,
loading: false,
selectedTime: '15m',
isAutoRefreshDisabled: true,
selectedAutoRefreshInterval: 'off',
...overrides,
});
beforeEach(() => {
// Reset Zustand store before each test
const { result } = renderHook(() => useGlobalTimeStore());
act(() => {
result.current.setSelectedTime(DEFAULT_TIME_RANGE, 0);
});
});
it('should render null because it just an adapter', () => {
store = mockStore({
globalTime: createGlobalTimeState(),
});
const { container } = render(
<Provider store={store}>
<GlobalTimeStoreAdapter />
</Provider>,
);
expect(container.firstChild).toBeNull();
});
it('should sync relative time from Redux to Zustand store', () => {
store = mockStore({
globalTime: createGlobalTimeState({
selectedTime: '15m',
isAutoRefreshDisabled: true,
selectedAutoRefreshInterval: 'off',
}),
});
render(
<Provider store={store}>
<GlobalTimeStoreAdapter />
</Provider>,
);
const { result } = renderHook(() => useGlobalTimeStore());
expect(result.current.selectedTime).toBe('15m');
expect(result.current.refreshInterval).toBe(0);
expect(result.current.isRefreshEnabled).toBe(false);
});
it('should sync custom time from Redux to Zustand store', () => {
store = mockStore({
globalTime: createGlobalTimeState({
selectedTime: 'custom',
minTime: randomTime,
maxTime: randomTime,
isAutoRefreshDisabled: true,
}),
});
render(
<Provider store={store}>
<GlobalTimeStoreAdapter />
</Provider>,
);
const { result } = renderHook(() => useGlobalTimeStore());
expect(result.current.selectedTime).toBe(
createCustomTimeRange(randomTime, randomTime),
);
expect(result.current.isRefreshEnabled).toBe(false);
});
it('should sync refresh interval when auto refresh is enabled', () => {
store = mockStore({
globalTime: createGlobalTimeState({
selectedTime: '15m',
isAutoRefreshDisabled: false,
selectedAutoRefreshInterval: '5s',
}),
});
render(
<Provider store={store}>
<GlobalTimeStoreAdapter />
</Provider>,
);
const { result } = renderHook(() => useGlobalTimeStore());
expect(result.current.selectedTime).toBe('15m');
expect(result.current.refreshInterval).toBe(5000); // 5s = 5000ms
expect(result.current.isRefreshEnabled).toBe(true);
});
it('should set refreshInterval to 0 when auto refresh is disabled', () => {
store = mockStore({
globalTime: createGlobalTimeState({
selectedTime: '15m',
isAutoRefreshDisabled: true,
selectedAutoRefreshInterval: '5s', // Even with interval set, should be 0 when disabled
}),
});
render(
<Provider store={store}>
<GlobalTimeStoreAdapter />
</Provider>,
);
const { result } = renderHook(() => useGlobalTimeStore());
expect(result.current.refreshInterval).toBe(0);
expect(result.current.isRefreshEnabled).toBe(false);
});
it('should update Zustand store when Redux state changes', () => {
store = mockStore({
globalTime: createGlobalTimeState({
selectedTime: '15m',
isAutoRefreshDisabled: true,
}),
});
const { rerender } = render(
<Provider store={store}>
<GlobalTimeStoreAdapter />
</Provider>,
);
// Verify initial state
let zustandState = renderHook(() => useGlobalTimeStore());
expect(zustandState.result.current.selectedTime).toBe('15m');
// Update Redux store
const newStore = mockStore({
globalTime: createGlobalTimeState({
selectedTime: '1h',
isAutoRefreshDisabled: false,
selectedAutoRefreshInterval: '30s',
}),
});
rerender(
<Provider store={newStore}>
<GlobalTimeStoreAdapter />
</Provider>,
);
// Verify updated state
zustandState = renderHook(() => useGlobalTimeStore());
expect(zustandState.result.current.selectedTime).toBe('1h');
expect(zustandState.result.current.refreshInterval).toBe(30000); // 30s = 30000ms
expect(zustandState.result.current.isRefreshEnabled).toBe(true);
});
it('should handle various refresh interval options', () => {
const testCases = [
{ key: '5s', expectedValue: 5000 },
{ key: '10s', expectedValue: 10000 },
{ key: '30s', expectedValue: 30000 },
{ key: '1m', expectedValue: 60000 },
{ key: '5m', expectedValue: 300000 },
];
testCases.forEach(({ key, expectedValue }) => {
store = mockStore({
globalTime: createGlobalTimeState({
selectedTime: '15m',
isAutoRefreshDisabled: false,
selectedAutoRefreshInterval: key,
}),
});
render(
<Provider store={store}>
<GlobalTimeStoreAdapter />
</Provider>,
);
const { result } = renderHook(() => useGlobalTimeStore());
expect(result.current.refreshInterval).toBe(expectedValue);
});
});
it('should handle unknown refresh interval by setting 0', () => {
store = mockStore({
globalTime: createGlobalTimeState({
selectedTime: '15m',
isAutoRefreshDisabled: false,
selectedAutoRefreshInterval: 'unknown-interval',
}),
});
render(
<Provider store={store}>
<GlobalTimeStoreAdapter />
</Provider>,
);
const { result } = renderHook(() => useGlobalTimeStore());
expect(result.current.refreshInterval).toBe(0);
expect(result.current.isRefreshEnabled).toBe(false);
});
});

View File

@@ -28,22 +28,14 @@
}
}
// In table/column view, keep action buttons visible at the viewport's right edge
.log-line-action-buttons.table-view-log-actions {
position: absolute;
top: 50%;
right: 8px;
left: auto;
transform: translateY(-50%);
margin: 0;
z-index: 5;
}
.lightMode {
.log-line-action-buttons {
border: 1px solid var(--bg-vanilla-400);
background: var(--bg-vanilla-400);
.ant-btn-default {
}
.copy-log-btn {
border-left: 1px solid var(--bg-vanilla-400);
border-color: var(--bg-vanilla-400) !important;

View File

@@ -15,12 +15,13 @@ export function getDefaultCellStyle(isDarkMode?: boolean): CSSProperties {
letterSpacing: '-0.07px',
marginBottom: '0px',
minWidth: '10rem',
width: 'auto',
width: '10rem',
};
}
export const defaultTableStyle: CSSProperties = {
minWidth: '40rem',
maxWidth: '90rem',
};
export const defaultListViewPanelStyle: CSSProperties = {

View File

@@ -33,7 +33,7 @@
display: flex;
align-items: center;
p {
.ant-typography {
margin-bottom: 0;
}
}
@@ -45,7 +45,6 @@
}
.paragraph {
margin: 0;
padding: 0px !important;
&.small {
font-size: 11px !important;

View File

@@ -1,5 +1,5 @@
import { useMemo } from 'react';
import { TableColumnsType as ColumnsType } from 'antd';
import { TableColumnsType as ColumnsType, Typography } from 'antd';
import cx from 'classnames';
import { DATE_TIME_FORMATS } from 'constants/dateTimeFormats';
import { getSanitizedLogBody } from 'container/LogDetailedView/utils';
@@ -43,7 +43,7 @@ export const useTableView = (props: UseTableViewProps): UseTableViewResult => {
const bodyColumnStyle = useMemo(
() => ({
...defaultTableStyle,
...(fields.length > 2 ? { width: 'auto' } : {}),
...(fields.length > 2 ? { width: '50rem' } : {}),
}),
[fields.length],
);
@@ -59,18 +59,18 @@ export const useTableView = (props: UseTableViewProps): UseTableViewResult => {
key: name,
render: (field): ColumnTypeRender<Record<string, unknown>> => ({
props: {
style: {
...(isListViewPanel
? defaultListViewPanelStyle
: getDefaultCellStyle(isDarkMode)),
display: '-webkit-box',
WebkitLineClamp: linesPerRow,
WebkitBoxOrient: 'vertical',
overflow: 'hidden',
wordBreak: 'break-all',
},
style: isListViewPanel
? defaultListViewPanelStyle
: getDefaultCellStyle(isDarkMode),
},
children: <p className={cx('paragraph', fontSize)}>{field}</p>,
children: (
<Typography.Paragraph
ellipsis={{ rows: linesPerRow }}
className={cx('paragraph', fontSize)}
>
{field}
</Typography.Paragraph>
),
}),
}));
@@ -123,7 +123,9 @@ export const useTableView = (props: UseTableViewProps): UseTableViewResult => {
return {
children: (
<div className="table-timestamp">
<p className={cx('text', fontSize)}>{date}</p>
<Typography.Paragraph ellipsis className={cx('text', fontSize)}>
{date}
</Typography.Paragraph>
</div>
),
};

View File

@@ -1,35 +1,21 @@
.quick-filters-container {
display: flex;
flex-direction: row;
height: 100%;
min-height: 0;
position: relative;
.quick-filters-settings-container {
flex: 0 0 0;
width: 0;
min-width: 0;
overflow: visible;
position: relative;
align-self: stretch;
}
}
.quick-filters {
display: flex;
flex-direction: column;
flex: 1 1 auto;
min-width: 0;
height: 100%;
min-height: 0;
width: 100%;
border-right: 1px solid var(--bg-slate-400);
color: var(--bg-vanilla-100);
.overlay-scrollbar {
flex: 1;
min-height: 0;
}
.header {
display: flex;
align-items: center;

View File

@@ -2,8 +2,6 @@
display: flex;
flex-direction: column;
position: absolute;
top: 0;
left: 0;
z-index: 999;
width: 342px;
background: var(--bg-slate-500);

View File

@@ -12,7 +12,6 @@ export enum LOCALSTORAGE {
GRAPH_VISIBILITY_STATES = 'GRAPH_VISIBILITY_STATES',
TRACES_LIST_COLUMNS = 'TRACES_LIST_COLUMNS',
LOGS_LIST_COLUMNS = 'LOGS_LIST_COLUMNS',
LOGS_LIST_COLUMN_SIZING = 'LOGS_LIST_COLUMN_SIZING',
LOGGED_IN_USER_NAME = 'LOGGED_IN_USER_NAME',
LOGGED_IN_USER_EMAIL = 'LOGGED_IN_USER_EMAIL',
CHAT_SUPPORT = 'CHAT_SUPPORT',

View File

@@ -1,11 +1,4 @@
export const REACT_QUERY_KEY = {
/**
* For any query that should support AutoRefresh and min/max time is from DateTimeSelectionV2
* You can prefix the query with this KEY, it will allow the queries to be automatically refreshed
* when the user clicks in the refresh button, or alert the user when the data is being refreshed.
*/
AUTO_REFRESH_QUERY: 'AUTO_REFRESH_QUERY',
GET_PUBLIC_DASHBOARD: 'GET_PUBLIC_DASHBOARD',
GET_PUBLIC_DASHBOARD_META: 'GET_PUBLIC_DASHBOARD_META',
GET_PUBLIC_DASHBOARD_WIDGET_DATA: 'GET_PUBLIC_DASHBOARD_WIDGET_DATA',

View File

@@ -1,16 +1,3 @@
.live-logs-container {
display: flex;
flex: 1;
min-height: 0;
}
.live-logs-content {
display: flex;
flex: 1;
flex-direction: column;
min-height: 0;
}
.live-logs-chart-container {
height: 200px;
min-height: 200px;
@@ -18,12 +5,6 @@
border-right: none;
}
.live-logs-list-container {
display: flex;
flex: 1;
min-height: 0;
}
.live-logs-settings-panel {
display: flex;
align-items: center;

View File

@@ -18,19 +18,6 @@
}
.live-logs-list {
display: flex;
flex: 1;
flex-direction: column;
min-height: 0;
.ant-card,
.ant-card-body {
display: flex;
flex: 1;
min-height: 0;
flex-direction: column;
}
.live-logs-list-loading {
padding: 16px;
text-align: center;

View File

@@ -8,8 +8,8 @@ import OverlayScrollbar from 'components/OverlayScrollbar/OverlayScrollbar';
import { CARD_BODY_STYLE } from 'constants/card';
import { LOCALSTORAGE } from 'constants/localStorage';
import { OptionFormatTypes } from 'constants/optionsFormatTypes';
import InfinityTableView from 'container/LogsExplorerList/InfinityTableView';
import { InfinityWrapperStyled } from 'container/LogsExplorerList/styles';
import TanStackTableView from 'container/LogsExplorerList/TanStackTableView';
import { convertKeysToColumnFields } from 'container/LogsExplorerList/utils';
import { useOptionsMenu } from 'container/OptionsMenu';
import { defaultLogsSelectedColumns } from 'container/OptionsMenu/constants';
@@ -50,7 +50,7 @@ function LiveLogsList({
[logs],
);
const { options, config } = useOptionsMenu({
const { options } = useOptionsMenu({
storageKey: LOCALSTORAGE.LOGS_LIST_OPTIONS,
dataSource: DataSource.LOGS,
aggregateOperator: StringOperators.NOOP,
@@ -158,7 +158,7 @@ function LiveLogsList({
{formattedLogs.length !== 0 && (
<InfinityWrapperStyled>
{options.format === OptionFormatTypes.TABLE ? (
<TanStackTableView
<InfinityTableView
ref={ref}
isLoading={false}
tableViewProps={{
@@ -174,7 +174,6 @@ function LiveLogsList({
onSetActiveLog={handleSetActiveLog}
onClearActiveLog={handleCloseLogDetail}
activeLog={activeLog}
onRemoveColumn={config.addColumn?.onRemove}
/>
) : (
<Card style={{ width: '100%' }} bodyStyle={CARD_BODY_STYLE}>

View File

@@ -4,11 +4,18 @@ import { FontSize } from 'container/OptionsMenu/types';
export const infinityDefaultStyles: CSSProperties = {
width: '100%',
overflowX: 'scroll',
marginTop: '15px',
};
export function getInfinityDefaultStyles(_fontSize: FontSize): CSSProperties {
export function getInfinityDefaultStyles(fontSize: FontSize): CSSProperties {
return {
width: '100%',
overflowX: 'scroll',
marginTop:
fontSize === FontSize.SMALL
? '10px'
: fontSize === FontSize.MEDIUM
? '12px'
: '15px',
};
}

View File

@@ -14,78 +14,6 @@ interface TableHeaderCellStyledProps {
export const TableStyled = styled.table`
width: 100%;
border-collapse: separate;
border-spacing: 0;
`;
/**
* TanStack column sizing uses table-layout:fixed + colgroup widths; without clipping,
* cell content overflows visually on top of neighbouring columns (overlap / "ghost" text).
*/
export const TanStackTableStyled = styled(TableStyled)`
table-layout: fixed;
width: 100%;
min-width: 100%;
max-width: 100%;
& td,
& th {
overflow: hidden;
min-width: 0;
box-sizing: border-box;
vertical-align: middle;
}
& td.table-actions-cell {
overflow: visible;
}
& td.body {
word-break: break-word;
overflow-wrap: anywhere;
}
/* Let nested body HTML / line-clamp shrink inside fixed columns */
& td.body > * {
min-width: 0;
max-width: 100%;
}
/* Long column titles: ellipsis when wider than the column (TanStackHeaderRow) */
& thead th .tanstack-header-title {
min-width: 0;
flex: 1 1 0;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
& thead th .tanstack-header-title > * {
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
max-width: 100%;
}
& td.logs-table-filler-cell,
& th.logs-table-filler-header {
padding: 0 !important;
min-width: 0;
border-left: none;
}
& th.logs-table-actions-header {
position: sticky;
right: 0;
z-index: 2;
width: 0 !important;
min-width: 0 !important;
max-width: 0 !important;
padding: 0 !important;
overflow: visible;
white-space: nowrap;
border-left: none;
}
`;
const getTimestampColumnWidth = (
@@ -118,19 +46,6 @@ export const TableCellStyled = styled.td<TableHeaderCellStyledProps>`
${({ columnKey, $hasSingleColumn }): string =>
getTimestampColumnWidth(columnKey, $hasSingleColumn)}
&.table-actions-cell {
position: sticky;
right: 0;
z-index: 2;
width: 0;
min-width: 0;
max-width: 0;
padding: 0 !important;
white-space: nowrap;
overflow: visible;
background-color: inherit;
}
`;
export const TableRowStyled = styled.tr<{
@@ -149,10 +64,7 @@ export const TableRowStyled = styled.tr<{
position: relative;
.log-line-action-buttons {
display: flex;
opacity: 0;
pointer-events: none;
transition: opacity 80ms linear;
display: none;
}
&:hover {
@@ -161,25 +73,13 @@ export const TableRowStyled = styled.tr<{
getActiveLogBackground(true, $isDarkMode, $logType)}
}
.log-line-action-buttons {
opacity: 1;
pointer-events: auto;
display: flex;
}
}
${({ $isActiveLog }): string =>
$isActiveLog
? `
.log-line-action-buttons {
opacity: 1;
pointer-events: auto;
}
`
: ''}
`;
export const TableHeaderCellStyled = styled.th<TableHeaderCellStyledProps>`
padding: 0.5rem;
height: 36px;
text-align: left;
font-size: 14px;
font-style: normal;
font-weight: 400;
@@ -198,12 +98,6 @@ export const TableHeaderCellStyled = styled.th<TableHeaderCellStyledProps>`
: ``};
${({ $isLogIndicator }): string =>
$isLogIndicator ? 'padding: 0px; width: 1%;' : ''}
border-top: 1px solid var(--l2-border);
border-bottom: 1px solid var(--l2-border);
box-shadow: inset 0 -1px 0 var(--l2-border);
&:first-child {
border-left: 1px solid var(--l2-border);
}
color: ${(props): string =>
props.$isDarkMode ? 'var(--bg-vanilla-100, #fff)' : themeColors.bckgGrey};

View File

@@ -5,8 +5,6 @@ import { ILog } from 'types/api/logs/log';
export type InfinityTableProps = {
isLoading?: boolean;
isFetching?: boolean;
onRemoveColumn?: (columnKey: string) => void;
tableViewProps: Omit<UseTableViewProps, 'onOpenLogsContext' | 'onClickExpand'>;
infitiyTableProps?: {
onEndReached: (index: number) => void;

View File

@@ -8,7 +8,7 @@
line-height: 18px;
letter-spacing: -0.005em;
text-align: left;
min-height: 0;
min-height: 500px;
.logs-list-table-view-container {
.data-table-container {
@@ -24,11 +24,11 @@
color: white !important;
.cursor-col-resize {
width: 24px !important;
width: 3px !important;
cursor: col-resize !important;
opacity: 1 !important;
background-color: transparent !important;
border: none !important;
opacity: 0.5 !important;
background-color: var(--bg-ink-500) !important;
border: 1px solid var(--bg-ink-500) !important;
&:hover {
opacity: 1 !important;
@@ -85,7 +85,7 @@
}
thead {
z-index: 2 !important;
z-index: 0 !important;
}
.log-state-indicator {

View File

@@ -1,8 +0,0 @@
// eslint-disable-next-line no-restricted-imports
import { createContext, useContext } from 'react';
const RowHoverContext = createContext(false);
export const useRowHover = (): boolean => useContext(RowHoverContext);
export default RowHoverContext;

View File

@@ -1,84 +0,0 @@
import { ComponentProps, memo, useCallback, useState } from 'react';
import { TableComponents } from 'react-virtuoso';
import {
getLogIndicatorType,
getLogIndicatorTypeForTable,
} from 'components/Logs/LogStateIndicator/utils';
import { useCopyLogLink } from 'hooks/logs/useCopyLogLink';
import { useIsDarkMode } from 'hooks/useDarkMode';
import { ILog } from 'types/api/logs/log';
import { TableRowStyled } from '../InfinityTableView/styles';
import RowHoverContext from '../RowHoverContext';
import { TanStackTableRowData } from './types';
export type TableRowContext = {
activeLog?: ILog | null;
activeContextLog?: ILog | null;
logsById: Map<string, ILog>;
};
type VirtuosoTableRowProps = ComponentProps<
NonNullable<TableComponents<TanStackTableRowData, TableRowContext>['TableRow']>
>;
type TanStackCustomTableRowProps = VirtuosoTableRowProps;
function TanStackCustomTableRow({
children,
item,
context,
...props
}: TanStackCustomTableRowProps): JSX.Element {
const { isHighlighted } = useCopyLogLink(item.currentLog.id);
const isDarkMode = useIsDarkMode();
const [hasHovered, setHasHovered] = useState(false);
const rowId = String(item.currentLog.id ?? '');
const activeLog = context?.activeLog;
const activeContextLog = context?.activeContextLog;
const logsById = context?.logsById;
const rowLog = logsById?.get(rowId) || item.currentLog;
const logType = rowLog
? getLogIndicatorType(rowLog)
: getLogIndicatorTypeForTable(item.log);
const handleMouseEnter = useCallback(() => {
if (!hasHovered) {
setHasHovered(true);
}
}, [hasHovered]);
return (
<RowHoverContext.Provider value={hasHovered}>
<TableRowStyled
{...props}
$isDarkMode={isDarkMode}
$isActiveLog={
isHighlighted ||
rowId === String(activeLog?.id ?? '') ||
rowId === String(activeContextLog?.id ?? '')
}
$logType={logType}
onMouseEnter={handleMouseEnter}
>
{children}
</TableRowStyled>
</RowHoverContext.Provider>
);
}
export default memo(TanStackCustomTableRow, (prev, next) => {
const prevId = String(prev.item.currentLog.id ?? '');
const nextId = String(next.item.currentLog.id ?? '');
if (prevId !== nextId) {
return false;
}
const prevIsActive =
prevId === String(prev.context?.activeLog?.id ?? '') ||
prevId === String(prev.context?.activeContextLog?.id ?? '');
const nextIsActive =
nextId === String(next.context?.activeLog?.id ?? '') ||
nextId === String(next.context?.activeContextLog?.id ?? '');
return prevIsActive === nextIsActive;
});

View File

@@ -1,193 +0,0 @@
import type {
CSSProperties,
MouseEvent as ReactMouseEvent,
TouchEvent as ReactTouchEvent,
} from 'react';
import { useMemo } from 'react';
import { CloseOutlined, MoreOutlined } from '@ant-design/icons';
import { useSortable } from '@dnd-kit/sortable';
import { Popover, PopoverContent, PopoverTrigger } from '@signozhq/popover';
import { flexRender, Header as TanStackHeader } from '@tanstack/react-table';
import { GripVertical } from 'lucide-react';
import { TableHeaderCellStyled } from '../InfinityTableView/styles';
import { InfinityTableProps } from '../InfinityTableView/types';
import { OrderedColumn, TanStackTableRowData } from './types';
import { getColumnId } from './utils';
import './styles/TanStackHeaderRow.styles.scss';
type TanStackHeaderRowProps = {
column: OrderedColumn;
header?: TanStackHeader<TanStackTableRowData, unknown>;
isDarkMode: boolean;
fontSize: InfinityTableProps['tableViewProps']['fontSize'];
hasSingleColumn: boolean;
canRemoveColumn?: boolean;
onRemoveColumn?: (columnKey: string) => void;
};
const GRIP_ICON_SIZE = 12;
// eslint-disable-next-line sonarjs/cognitive-complexity
function TanStackHeaderRow({
column,
header,
isDarkMode,
fontSize,
hasSingleColumn,
canRemoveColumn = false,
onRemoveColumn,
}: TanStackHeaderRowProps): JSX.Element {
const columnId = getColumnId(column);
const isDragColumn =
column.key !== 'expand' && column.key !== 'state-indicator';
const isResizableColumn = Boolean(header?.column.getCanResize());
const isColumnRemovable = Boolean(
canRemoveColumn &&
onRemoveColumn &&
column.key !== 'expand' &&
column.key !== 'state-indicator',
);
const isResizing = Boolean(header?.column.getIsResizing());
const resizeHandler = header?.getResizeHandler();
const headerText =
typeof column.title === 'string' && column.title
? column.title
: String(header?.id ?? columnId);
const headerTitleAttr = headerText.replace(/^\w/, (c) => c.toUpperCase());
const handleResizeStart = (
event: ReactMouseEvent<HTMLElement> | ReactTouchEvent<HTMLElement>,
): void => {
event.preventDefault();
event.stopPropagation();
resizeHandler?.(event);
};
const {
attributes,
listeners,
setNodeRef,
setActivatorNodeRef,
transform,
transition,
isDragging,
} = useSortable({
id: columnId,
disabled: !isDragColumn,
});
const headerCellStyle = useMemo(
() =>
({
'--tanstack-header-translate-x': `${Math.round(transform?.x ?? 0)}px`,
'--tanstack-header-translate-y': `${Math.round(transform?.y ?? 0)}px`,
'--tanstack-header-transition': isResizing ? 'none' : transition || 'none',
} as CSSProperties),
[isResizing, transform?.x, transform?.y, transition],
);
const headerCellClassName = [
'tanstack-header-cell',
isDragging ? 'is-dragging' : '',
isResizing ? 'is-resizing' : '',
]
.filter(Boolean)
.join(' ');
const headerContentClassName = [
'tanstack-header-content',
isResizableColumn ? 'has-resize-control' : '',
isColumnRemovable ? 'has-action-control' : '',
]
.filter(Boolean)
.join(' ');
return (
<TableHeaderCellStyled
ref={setNodeRef}
$isLogIndicator={column.key === 'state-indicator'}
$isDarkMode={isDarkMode}
$isDragColumn={false}
className={headerCellClassName}
key={columnId}
fontSize={fontSize}
$hasSingleColumn={hasSingleColumn}
style={headerCellStyle}
>
<span className={headerContentClassName}>
{isDragColumn ? (
<span className="tanstack-grip-slot">
<span
ref={setActivatorNodeRef}
{...attributes}
{...listeners}
role="button"
aria-label={`Drag ${String(
column.title || header?.id || columnId,
)} column`}
className="tanstack-grip-activator"
>
<GripVertical size={GRIP_ICON_SIZE} />
</span>
</span>
) : null}
<span className="tanstack-header-title" title={headerTitleAttr}>
{header
? flexRender(header.column.columnDef.header, header.getContext())
: String(column.title || '').replace(/^\w/, (c) => c.toUpperCase())}
</span>
{isColumnRemovable && (
<Popover>
<PopoverTrigger asChild>
<span
role="button"
aria-label={`Column actions for ${headerTitleAttr}`}
className="tanstack-header-action-trigger"
onMouseDown={(event): void => {
event.stopPropagation();
}}
>
<MoreOutlined />
</span>
</PopoverTrigger>
<PopoverContent
align="end"
sideOffset={6}
className="tanstack-column-actions-content"
>
<button
type="button"
className="tanstack-remove-column-action"
onClick={(event): void => {
event.preventDefault();
event.stopPropagation();
onRemoveColumn?.(String(column.key));
}}
>
<CloseOutlined className="tanstack-remove-column-action-icon" />
Remove column
</button>
</PopoverContent>
</Popover>
)}
</span>
{isResizableColumn && (
<span
role="presentation"
className="cursor-col-resize"
title="Drag to resize column"
onClick={(event): void => {
event.preventDefault();
event.stopPropagation();
}}
onMouseDown={(event): void => {
handleResizeStart(event);
}}
onTouchStart={(event): void => {
handleResizeStart(event);
}}
>
<span className="tanstack-resize-handle-line" />
</span>
)}
</TableHeaderCellStyled>
);
}
export default TanStackHeaderRow;

View File

@@ -1,95 +0,0 @@
import {
MouseEvent as ReactMouseEvent,
MouseEventHandler,
useCallback,
} from 'react';
import { flexRender, Row as TanStackRowModel } from '@tanstack/react-table';
import { VIEW_TYPES } from 'components/LogDetail/constants';
import LogLinesActionButtons from 'components/Logs/LogLinesActionButtons/LogLinesActionButtons';
import { TableCellStyled } from '../InfinityTableView/styles';
import { InfinityTableProps } from '../InfinityTableView/types';
import { useRowHover } from '../RowHoverContext';
import { TanStackTableRowData } from './types';
type TanStackRowCellsProps = {
row: TanStackRowModel<TanStackTableRowData>;
fontSize: InfinityTableProps['tableViewProps']['fontSize'];
onSetActiveLog?: InfinityTableProps['onSetActiveLog'];
onClearActiveLog?: InfinityTableProps['onClearActiveLog'];
isActiveLog?: boolean;
isDarkMode: boolean;
onLogCopy: (logId: string, event: ReactMouseEvent<HTMLElement>) => void;
isLogsExplorerPage: boolean;
};
function TanStackRowCells({
row,
fontSize,
onSetActiveLog,
onClearActiveLog,
isActiveLog = false,
isDarkMode,
onLogCopy,
isLogsExplorerPage,
}: TanStackRowCellsProps): JSX.Element {
const { currentLog } = row.original;
const hasHovered = useRowHover();
const handleShowContext: MouseEventHandler<HTMLElement> = useCallback(
(event) => {
event.preventDefault();
event.stopPropagation();
onSetActiveLog?.(currentLog, VIEW_TYPES.CONTEXT);
},
[currentLog, onSetActiveLog],
);
const handleShowLogDetails = useCallback(() => {
if (!currentLog) {
return;
}
if (isActiveLog && onClearActiveLog) {
onClearActiveLog();
return;
}
onSetActiveLog?.(currentLog);
}, [currentLog, isActiveLog, onClearActiveLog, onSetActiveLog]);
const visibleCells = row.getVisibleCells();
const lastCellIndex = visibleCells.length - 1;
return (
<>
{visibleCells.map((cell, index) => {
const columnKey = cell.column.id;
const isLastCell = index === lastCellIndex;
return (
<TableCellStyled
$isDragColumn={false}
$isLogIndicator={columnKey === 'state-indicator'}
$hasSingleColumn={visibleCells.length <= 2}
$isDarkMode={isDarkMode}
key={cell.id}
fontSize={fontSize}
className={columnKey}
onClick={handleShowLogDetails}
>
{flexRender(cell.column.columnDef.cell, cell.getContext())}
{isLastCell && isLogsExplorerPage && hasHovered && (
<LogLinesActionButtons
handleShowContext={handleShowContext}
onLogCopy={(event): void => onLogCopy(currentLog.id, event)}
customClassName="table-view-log-actions"
/>
)}
</TableCellStyled>
);
})}
</>
);
}
export default TanStackRowCells;

View File

@@ -1,105 +0,0 @@
import { render, screen } from '@testing-library/react';
import TanStackCustomTableRow, {
TableRowContext,
} from '../TanStackCustomTableRow';
import type { TanStackTableRowData } from '../types';
jest.mock('../../InfinityTableView/styles', () => ({
TableRowStyled: 'tr',
}));
jest.mock('hooks/logs/useCopyLogLink', () => ({
useCopyLogLink: (): { isHighlighted: boolean } => ({ isHighlighted: false }),
}));
jest.mock('hooks/useDarkMode', () => ({
useIsDarkMode: (): boolean => false,
}));
jest.mock('components/Logs/LogStateIndicator/utils', () => ({
getLogIndicatorType: (): string => 'info',
getLogIndicatorTypeForTable: (): string => 'info',
}));
const item: TanStackTableRowData = {
log: {},
currentLog: { id: 'row-1' } as TanStackTableRowData['currentLog'],
rowIndex: 0,
};
const virtuosoTableRowAttrs = {
'data-index': 0,
'data-item-index': 0,
'data-known-size': 40,
} as const;
const defaultContext: TableRowContext = {
activeLog: null,
activeContextLog: null,
logsById: new Map(),
};
describe('TanStackCustomTableRow', () => {
it('renders children inside TableRowStyled', () => {
render(
<table>
<tbody>
<TanStackCustomTableRow
{...virtuosoTableRowAttrs}
item={item}
context={defaultContext}
>
<td>cell</td>
</TanStackCustomTableRow>
</tbody>
</table>,
);
expect(screen.getByText('cell')).toBeInTheDocument();
});
it('marks row active when activeLog matches item id', () => {
const { container } = render(
<table>
<tbody>
<TanStackCustomTableRow
{...virtuosoTableRowAttrs}
item={item}
context={{
...defaultContext,
activeLog: { id: 'row-1' } as never,
}}
>
<td>x</td>
</TanStackCustomTableRow>
</tbody>
</table>,
);
const row = container.querySelector('tr');
expect(row).toBeTruthy();
});
it('uses logsById entry when present for indicator type', () => {
const logFromMap = { id: 'row-1', severity_text: 'error' } as never;
render(
<table>
<tbody>
<TanStackCustomTableRow
{...virtuosoTableRowAttrs}
item={item}
context={{
...defaultContext,
logsById: new Map([['row-1', logFromMap]]),
}}
>
<td>x</td>
</TanStackCustomTableRow>
</tbody>
</table>,
);
expect(screen.getByText('x')).toBeInTheDocument();
});
});

View File

@@ -1,152 +0,0 @@
import type { Header } from '@tanstack/react-table';
import { render, screen } from '@testing-library/react';
import { FontSize } from 'container/OptionsMenu/types';
import TanStackHeaderRow from '../TanStackHeaderRow';
import type { OrderedColumn, TanStackTableRowData } from '../types';
jest.mock('../../InfinityTableView/styles', () => ({
TableHeaderCellStyled: 'th',
}));
const mockUseSortable = jest.fn((_args?: unknown) => ({
attributes: {},
listeners: {},
setNodeRef: jest.fn(),
setActivatorNodeRef: jest.fn(),
transform: null,
transition: undefined,
isDragging: false,
}));
jest.mock('@dnd-kit/sortable', () => ({
useSortable: (args: unknown): ReturnType<typeof mockUseSortable> =>
mockUseSortable(args),
}));
jest.mock('@tanstack/react-table', () => ({
flexRender: (def: unknown, ctx: unknown): unknown => {
if (typeof def === 'string') {
return def;
}
if (typeof def === 'function') {
return (def as (c: unknown) => unknown)(ctx);
}
return def;
},
}));
const column = (key: string): OrderedColumn =>
({ key, title: key } as OrderedColumn);
const mockHeader = (
id: string,
canResize = true,
): Header<TanStackTableRowData, unknown> =>
(({
id,
column: {
getCanResize: (): boolean => canResize,
getIsResizing: (): boolean => false,
columnDef: { header: id },
},
getContext: (): unknown => ({}),
getResizeHandler: (): (() => void) => jest.fn(),
flexRender: undefined,
} as unknown) as Header<TanStackTableRowData, unknown>);
describe('TanStackHeaderRow', () => {
beforeEach(() => {
mockUseSortable.mockClear();
});
it('renders column title when header is undefined', () => {
render(
<table>
<thead>
<tr>
<TanStackHeaderRow
column={column('timestamp')}
isDarkMode={false}
fontSize={FontSize.SMALL}
hasSingleColumn={false}
/>
</tr>
</thead>
</table>,
);
expect(screen.getByText('Timestamp')).toBeInTheDocument();
});
it('enables useSortable for draggable columns', () => {
render(
<table>
<thead>
<tr>
<TanStackHeaderRow
column={column('body')}
header={mockHeader('body')}
isDarkMode={false}
fontSize={FontSize.SMALL}
hasSingleColumn={false}
/>
</tr>
</thead>
</table>,
);
expect(mockUseSortable).toHaveBeenCalledWith(
expect.objectContaining({
id: 'body',
disabled: false,
}),
);
});
it('disables sortable for expand column', () => {
render(
<table>
<thead>
<tr>
<TanStackHeaderRow
column={column('expand')}
header={mockHeader('expand', false)}
isDarkMode={false}
fontSize={FontSize.SMALL}
hasSingleColumn={false}
/>
</tr>
</thead>
</table>,
);
expect(mockUseSortable).toHaveBeenCalledWith(
expect.objectContaining({
disabled: true,
}),
);
});
it('shows drag grip for draggable columns', () => {
render(
<table>
<thead>
<tr>
<TanStackHeaderRow
column={column('body')}
header={mockHeader('body')}
isDarkMode={false}
fontSize={FontSize.SMALL}
hasSingleColumn={false}
/>
</tr>
</thead>
</table>,
);
expect(
screen.getByRole('button', { name: /Drag body column/i }),
).toBeInTheDocument();
});
});

View File

@@ -1,193 +0,0 @@
import { fireEvent, render, screen } from '@testing-library/react';
import RowHoverContext from 'container/LogsExplorerList/RowHoverContext';
import { FontSize } from 'container/OptionsMenu/types';
import TanStackRowCells from '../TanStackRow';
import type { TanStackTableRowData } from '../types';
jest.mock('../../InfinityTableView/styles', () => ({
TableCellStyled: 'td',
}));
jest.mock(
'components/Logs/LogLinesActionButtons/LogLinesActionButtons',
() => ({
__esModule: true,
default: ({
onLogCopy,
}: {
onLogCopy: (e: React.MouseEvent) => void;
}): JSX.Element => (
<button type="button" data-testid="copy-btn" onClick={onLogCopy}>
copy
</button>
),
}),
);
const flexRenderMock = jest.fn((def: unknown, _ctx?: unknown) =>
typeof def === 'function' ? def({}) : def,
);
jest.mock('@tanstack/react-table', () => ({
flexRender: (def: unknown, ctx: unknown): unknown => flexRenderMock(def, ctx),
}));
function buildMockRow(
visibleCells: Array<{ columnId: string }>,
): Parameters<typeof TanStackRowCells>[0]['row'] {
return {
original: {
currentLog: { id: 'log-1' } as TanStackTableRowData['currentLog'],
log: {},
rowIndex: 0,
},
getVisibleCells: () =>
visibleCells.map((cell, index) => ({
id: `cell-${index}`,
column: {
id: cell.columnId,
columnDef: {
cell: (): string => `content-${cell.columnId}`,
},
},
getContext: (): Record<string, unknown> => ({}),
})),
} as never;
}
describe('TanStackRowCells', () => {
beforeEach(() => {
flexRenderMock.mockClear();
});
it('renders a cell per visible column and calls flexRender', () => {
const row = buildMockRow([
{ columnId: 'state-indicator' },
{ columnId: 'body' },
]);
render(
<table>
<tbody>
<tr>
<TanStackRowCells
row={row}
fontSize={FontSize.SMALL}
isDarkMode={false}
onLogCopy={jest.fn()}
isLogsExplorerPage={false}
/>
</tr>
</tbody>
</table>,
);
expect(screen.getAllByRole('cell')).toHaveLength(2);
expect(flexRenderMock).toHaveBeenCalled();
});
it('applies state-indicator styling class on the indicator cell', () => {
const row = buildMockRow([{ columnId: 'state-indicator' }]);
const { container } = render(
<table>
<tbody>
<tr>
<TanStackRowCells
row={row}
fontSize={FontSize.SMALL}
isDarkMode={false}
onLogCopy={jest.fn()}
isLogsExplorerPage={false}
/>
</tr>
</tbody>
</table>,
);
expect(container.querySelector('td.state-indicator')).toBeInTheDocument();
});
it('renders row actions on logs explorer page after hover', () => {
const row = buildMockRow([{ columnId: 'body' }]);
render(
<RowHoverContext.Provider value>
<table>
<tbody>
<tr>
<TanStackRowCells
row={row}
fontSize={FontSize.SMALL}
isDarkMode={false}
onLogCopy={jest.fn()}
isLogsExplorerPage
/>
</tr>
</tbody>
</table>
</RowHoverContext.Provider>,
);
expect(screen.getByTestId('copy-btn')).toBeInTheDocument();
});
it('click on a data cell calls onSetActiveLog with current log', () => {
const onSetActiveLog = jest.fn();
const row = buildMockRow([{ columnId: 'body' }]);
render(
<table>
<tbody>
<tr>
<TanStackRowCells
row={row}
fontSize={FontSize.SMALL}
isDarkMode={false}
onSetActiveLog={onSetActiveLog}
onLogCopy={jest.fn()}
isLogsExplorerPage={false}
/>
</tr>
</tbody>
</table>,
);
fireEvent.click(screen.getAllByRole('cell')[0]);
expect(onSetActiveLog).toHaveBeenCalledWith(
expect.objectContaining({ id: 'log-1' }),
);
});
it('when row is active log, click on cell clears active log', () => {
const onSetActiveLog = jest.fn();
const onClearActiveLog = jest.fn();
const row = buildMockRow([{ columnId: 'body' }]);
render(
<table>
<tbody>
<tr>
<TanStackRowCells
row={row}
fontSize={FontSize.SMALL}
isDarkMode={false}
isActiveLog
onSetActiveLog={onSetActiveLog}
onClearActiveLog={onClearActiveLog}
onLogCopy={jest.fn()}
isLogsExplorerPage={false}
/>
</tr>
</tbody>
</table>,
);
fireEvent.click(screen.getAllByRole('cell')[0]);
expect(onClearActiveLog).toHaveBeenCalled();
expect(onSetActiveLog).not.toHaveBeenCalled();
});
});

View File

@@ -1,104 +0,0 @@
import { forwardRef } from 'react';
import { render, screen } from '@testing-library/react';
import { FontSize } from 'container/OptionsMenu/types';
import type { InfinityTableProps } from '../../InfinityTableView/types';
import TanStackTableView from '../index';
jest.mock('react-virtuoso', () => ({
TableVirtuoso: forwardRef<
unknown,
{
fixedHeaderContent?: () => JSX.Element;
itemContent: (i: number) => JSX.Element;
}
>(function MockVirtuoso({ fixedHeaderContent, itemContent }, _ref) {
return (
<div data-testid="virtuoso">
{fixedHeaderContent?.()}
{itemContent(0)}
</div>
);
}),
}));
jest.mock('components/Logs/TableView/useTableView', () => ({
useTableView: (): {
dataSource: Record<string, string>[];
columns: unknown[];
} => ({
dataSource: [{ id: '1' }],
columns: [
{ key: 'body', title: 'body', render: (): string => 'x' },
{ key: 'state-indicator', title: 's', render: (): string => 'y' },
],
}),
}));
jest.mock('hooks/useDragColumns', () => ({
__esModule: true,
default: (): {
draggedColumns: unknown[];
onColumnOrderChange: () => void;
} => ({
draggedColumns: [],
onColumnOrderChange: jest.fn(),
}),
}));
jest.mock('hooks/logs/useActiveLog', () => ({
useActiveLog: (): { activeLog: null } => ({ activeLog: null }),
}));
jest.mock('hooks/logs/useCopyLogLink', () => ({
useCopyLogLink: (): { activeLogId: null } => ({ activeLogId: null }),
}));
jest.mock('hooks/useDarkMode', () => ({
useIsDarkMode: (): boolean => false,
}));
jest.mock('react-router-dom', () => ({
useLocation: (): { pathname: string } => ({ pathname: '/logs' }),
}));
jest.mock('react-use', () => ({
useCopyToClipboard: (): [unknown, () => void] => [null, jest.fn()],
}));
jest.mock('@signozhq/sonner', () => ({
toast: { success: jest.fn() },
}));
jest.mock('components/Spinner', () => ({
__esModule: true,
default: ({ tip }: { tip: string }): JSX.Element => (
<div data-testid="spinner">{tip}</div>
),
}));
const baseProps: InfinityTableProps = {
isLoading: false,
tableViewProps: {
logs: [{ id: '1' } as never],
fields: [],
linesPerRow: 3,
fontSize: FontSize.SMALL,
appendTo: 'end',
activeLogIndex: 0,
},
};
describe('TanStackTableView', () => {
it('shows spinner while loading', () => {
render(<TanStackTableView {...baseProps} isLoading />);
expect(screen.getByTestId('spinner')).toHaveTextContent('Getting Logs');
});
it('renders virtuoso when not loading', () => {
render(<TanStackTableView {...baseProps} />);
expect(screen.getByTestId('virtuoso')).toBeInTheDocument();
});
});

View File

@@ -1,173 +0,0 @@
import { act, renderHook } from '@testing-library/react';
import { LOCALSTORAGE } from 'constants/localStorage';
import type { OrderedColumn } from '../types';
import { useColumnSizingPersistence } from '../useColumnSizingPersistence';
const mockGet = jest.fn();
const mockSet = jest.fn();
jest.mock('api/browser/localstorage/get', () => ({
__esModule: true,
default: (key: string): string | null => mockGet(key),
}));
jest.mock('api/browser/localstorage/set', () => ({
__esModule: true,
default: (key: string, value: string): void => {
mockSet(key, value);
},
}));
const col = (key: string): OrderedColumn =>
({ key, title: key } as OrderedColumn);
describe('useColumnSizingPersistence', () => {
beforeEach(() => {
jest.clearAllMocks();
mockGet.mockReturnValue(null);
jest.useFakeTimers();
});
afterEach(() => {
jest.runOnlyPendingTimers();
jest.useRealTimers();
});
it('initializes with empty sizing when localStorage is empty', () => {
const { result } = renderHook(() =>
useColumnSizingPersistence([col('body'), col('timestamp')]),
);
expect(result.current.columnSizing).toEqual({});
});
it('parses flat ColumnSizingState from localStorage', () => {
mockGet.mockReturnValue(JSON.stringify({ body: 400, timestamp: 180 }));
const { result } = renderHook(() =>
useColumnSizingPersistence([col('body'), col('timestamp')]),
);
expect(result.current.columnSizing).toEqual({ body: 400, timestamp: 180 });
});
it('parses PersistedColumnSizing wrapper with sizing + columnIdsSignature', () => {
mockGet.mockReturnValue(
JSON.stringify({
version: 1,
columnIdsSignature: 'body|timestamp',
sizing: { body: 300 },
}),
);
const { result } = renderHook(() =>
useColumnSizingPersistence([col('body'), col('timestamp')]),
);
expect(result.current.columnSizing).toEqual({ body: 300 });
});
it('drops invalid numeric entries when reading from localStorage', () => {
mockGet.mockReturnValue(
JSON.stringify({
body: 200,
bad: NaN,
zero: 0,
neg: -1,
str: 'wide',
}),
);
const { result } = renderHook(() =>
useColumnSizingPersistence([col('body'), col('bad'), col('zero')]),
);
expect(result.current.columnSizing).toEqual({ body: 200 });
});
it('returns empty sizing when JSON is invalid', () => {
const spy = jest.spyOn(console, 'error').mockImplementation(() => {});
mockGet.mockReturnValue('not-json');
const { result } = renderHook(() =>
useColumnSizingPersistence([col('body')]),
);
expect(result.current.columnSizing).toEqual({});
spy.mockRestore();
});
it('prunes sizing for columns not in orderedColumns and strips fixed columns', () => {
mockGet.mockReturnValue(JSON.stringify({ body: 400, expand: 32, gone: 100 }));
const { result, rerender } = renderHook(
({ columns }: { columns: OrderedColumn[] }) =>
useColumnSizingPersistence(columns),
{
initialProps: {
columns: [
col('body'),
col('expand'),
col('state-indicator'),
] as OrderedColumn[],
},
},
);
expect(result.current.columnSizing).toEqual({ body: 400 });
act(() => {
rerender({
columns: [col('body'), col('expand'), col('state-indicator')],
});
});
expect(result.current.columnSizing).toEqual({ body: 400 });
});
it('updates setColumnSizing manually', () => {
const { result } = renderHook(() =>
useColumnSizingPersistence([col('body')]),
);
act(() => {
result.current.setColumnSizing({ body: 500 });
});
expect(result.current.columnSizing).toEqual({ body: 500 });
});
it('debounces writes to localStorage', () => {
const { result } = renderHook(() =>
useColumnSizingPersistence([col('body')]),
);
act(() => {
result.current.setColumnSizing({ body: 600 });
});
expect(mockSet).not.toHaveBeenCalled();
act(() => {
jest.advanceTimersByTime(250);
});
expect(mockSet).toHaveBeenCalledWith(
LOCALSTORAGE.LOGS_LIST_COLUMN_SIZING,
expect.stringContaining('"body":600'),
);
});
it('does not persist when ordered columns signature effect runs with empty ids early — still debounces empty sizing', () => {
const { result } = renderHook(() => useColumnSizingPersistence([]));
expect(result.current.columnSizing).toEqual({});
act(() => {
jest.advanceTimersByTime(250);
});
expect(mockSet).toHaveBeenCalled();
});
});

View File

@@ -1,222 +0,0 @@
import { act, renderHook } from '@testing-library/react';
import type { OrderedColumn } from '../types';
import { useOrderedColumns } from '../useOrderedColumns';
const mockGetDraggedColumns = jest.fn();
jest.mock('hooks/useDragColumns/utils', () => ({
getDraggedColumns: <T,>(current: unknown[], dragged: unknown[]): T[] =>
mockGetDraggedColumns(current, dragged) as T[],
}));
const col = (key: string, title?: string): OrderedColumn =>
({ key, title: title ?? key } as OrderedColumn);
describe('useOrderedColumns', () => {
beforeEach(() => {
jest.clearAllMocks();
});
it('returns columns from getDraggedColumns filtered to keys with string or number', () => {
mockGetDraggedColumns.mockReturnValue([
col('body'),
col('timestamp'),
{ title: 'no-key' },
]);
const { result } = renderHook(() =>
useOrderedColumns({
columns: [],
draggedColumns: [],
onColumnOrderChange: jest.fn(),
}),
);
expect(result.current.orderedColumns).toEqual([
col('body'),
col('timestamp'),
]);
expect(result.current.orderedColumnIds).toEqual(['body', 'timestamp']);
});
it('hasSingleColumn is true when exactly one column is not state-indicator', () => {
mockGetDraggedColumns.mockReturnValue([col('state-indicator'), col('body')]);
const { result } = renderHook(() =>
useOrderedColumns({
columns: [],
draggedColumns: [],
onColumnOrderChange: jest.fn(),
}),
);
expect(result.current.hasSingleColumn).toBe(true);
});
it('hasSingleColumn is false when more than one non-state-indicator column exists', () => {
mockGetDraggedColumns.mockReturnValue([
col('state-indicator'),
col('body'),
col('timestamp'),
]);
const { result } = renderHook(() =>
useOrderedColumns({
columns: [],
draggedColumns: [],
onColumnOrderChange: jest.fn(),
}),
);
expect(result.current.hasSingleColumn).toBe(false);
});
it('handleDragEnd reorders columns and calls onColumnOrderChange', () => {
const onColumnOrderChange = jest.fn();
mockGetDraggedColumns.mockReturnValue([col('a'), col('b'), col('c')]);
const { result } = renderHook(() =>
useOrderedColumns({
columns: [],
draggedColumns: [],
onColumnOrderChange,
}),
);
act(() => {
result.current.handleDragEnd({
active: { id: 'a' },
over: { id: 'c' },
} as never);
});
expect(onColumnOrderChange).toHaveBeenCalledWith([
expect.objectContaining({ key: 'b' }),
expect.objectContaining({ key: 'c' }),
expect.objectContaining({ key: 'a' }),
]);
// Derived-only: orderedColumns should remain until draggedColumns (URL/localStorage) updates.
expect(result.current.orderedColumns.map((c) => c.key)).toEqual([
'a',
'b',
'c',
]);
});
it('handleDragEnd no-ops when over is null', () => {
const onColumnOrderChange = jest.fn();
mockGetDraggedColumns.mockReturnValue([col('a'), col('b')]);
const { result } = renderHook(() =>
useOrderedColumns({
columns: [],
draggedColumns: [],
onColumnOrderChange,
}),
);
const before = result.current.orderedColumns;
act(() => {
result.current.handleDragEnd({
active: { id: 'a' },
over: null,
} as never);
});
expect(result.current.orderedColumns).toBe(before);
expect(onColumnOrderChange).not.toHaveBeenCalled();
});
it('handleDragEnd no-ops when active.id equals over.id', () => {
const onColumnOrderChange = jest.fn();
mockGetDraggedColumns.mockReturnValue([col('a'), col('b')]);
const { result } = renderHook(() =>
useOrderedColumns({
columns: [],
draggedColumns: [],
onColumnOrderChange,
}),
);
act(() => {
result.current.handleDragEnd({
active: { id: 'a' },
over: { id: 'a' },
} as never);
});
expect(onColumnOrderChange).not.toHaveBeenCalled();
});
it('handleDragEnd no-ops when indices cannot be resolved', () => {
const onColumnOrderChange = jest.fn();
mockGetDraggedColumns.mockReturnValue([col('a'), col('b')]);
const { result } = renderHook(() =>
useOrderedColumns({
columns: [],
draggedColumns: [],
onColumnOrderChange,
}),
);
act(() => {
result.current.handleDragEnd({
active: { id: 'missing' },
over: { id: 'a' },
} as never);
});
expect(onColumnOrderChange).not.toHaveBeenCalled();
});
it('exposes sensors from useSensors', () => {
mockGetDraggedColumns.mockReturnValue([col('a')]);
const { result } = renderHook(() =>
useOrderedColumns({
columns: [],
draggedColumns: [],
onColumnOrderChange: jest.fn(),
}),
);
expect(result.current.sensors).toBeDefined();
});
it('syncs ordered columns when base order changes externally (e.g. URL / localStorage)', () => {
mockGetDraggedColumns.mockReturnValue([col('a'), col('b'), col('c')]);
const { result, rerender } = renderHook(
({ draggedColumns }: { draggedColumns: unknown[] }) =>
useOrderedColumns({
columns: [],
draggedColumns,
onColumnOrderChange: jest.fn(),
}),
{ initialProps: { draggedColumns: [] as unknown[] } },
);
expect(result.current.orderedColumns.map((column) => column.key)).toEqual([
'a',
'b',
'c',
]);
mockGetDraggedColumns.mockReturnValue([col('c'), col('b'), col('a')]);
act(() => {
rerender({ draggedColumns: [{ title: 'from-url' }] as unknown[] });
});
expect(result.current.orderedColumns.map((column) => column.key)).toEqual([
'c',
'b',
'a',
]);
});
});

View File

@@ -1,433 +0,0 @@
import {
forwardRef,
memo,
MouseEvent as ReactMouseEvent,
ReactElement,
useCallback,
useEffect,
useImperativeHandle,
useMemo,
useRef,
useState,
} from 'react';
import { useLocation } from 'react-router-dom';
import { useCopyToClipboard } from 'react-use';
import { TableVirtuoso, TableVirtuosoHandle } from 'react-virtuoso';
import { DndContext, pointerWithin } from '@dnd-kit/core';
import {
horizontalListSortingStrategy,
SortableContext,
} from '@dnd-kit/sortable';
import { toast } from '@signozhq/sonner';
import {
ColumnDef,
getCoreRowModel,
useReactTable,
} from '@tanstack/react-table';
import { VIEW_TYPES } from 'components/LogDetail/constants';
import { ColumnTypeRender } from 'components/Logs/TableView/types';
import { useTableView } from 'components/Logs/TableView/useTableView';
import Spinner from 'components/Spinner';
import { LOCALSTORAGE } from 'constants/localStorage';
import { QueryParams } from 'constants/query';
import ROUTES from 'constants/routes';
import { useActiveLog } from 'hooks/logs/useActiveLog';
import { useIsDarkMode } from 'hooks/useDarkMode';
import useDragColumns from 'hooks/useDragColumns';
import { infinityDefaultStyles } from '../InfinityTableView/config';
import { TanStackTableStyled } from '../InfinityTableView/styles';
import { InfinityTableProps } from '../InfinityTableView/types';
import TanStackCustomTableRow from './TanStackCustomTableRow';
import TanStackHeaderRow from './TanStackHeaderRow';
import TanStackRowCells from './TanStackRow';
import { TableRecord, TanStackTableRowData } from './types';
import { useColumnSizingPersistence } from './useColumnSizingPersistence';
import { useOrderedColumns } from './useOrderedColumns';
import {
getColumnId,
getColumnMinWidthPx,
resolveColumnTypeRender,
} from './utils';
import '../logsTableVirtuosoScrollbar.scss';
import './styles/TanStackTableView.styles.scss';
const COLUMN_DND_AUTO_SCROLL = {
layoutShiftCompensation: false as const,
threshold: { x: 0.2, y: 0 },
};
const TanStackTableView = forwardRef<TableVirtuosoHandle, InfinityTableProps>(
function TanStackTableView(
{
isLoading,
isFetching,
onRemoveColumn,
tableViewProps,
infitiyTableProps,
onSetActiveLog,
onClearActiveLog,
activeLog,
}: InfinityTableProps,
forwardedRef,
): JSX.Element {
const { pathname } = useLocation();
const virtuosoRef = useRef<TableVirtuosoHandle | null>(null);
// could avoid this if directly use forwardedRef in TableVirtuoso, but need to verify if it causes any issue with react-virtuoso internal ref handling
useImperativeHandle(
forwardedRef,
() => virtuosoRef.current as TableVirtuosoHandle,
[],
);
const [, setCopy] = useCopyToClipboard();
const isDarkMode = useIsDarkMode();
const isLogsExplorerPage = pathname === ROUTES.LOGS_EXPLORER;
const { activeLog: activeContextLog } = useActiveLog();
// Column definitions (shared with existing logs table)
const { dataSource, columns } = useTableView({
...tableViewProps,
onClickExpand: onSetActiveLog,
onOpenLogsContext: (log): void => onSetActiveLog?.(log, VIEW_TYPES.CONTEXT),
});
// Column order (drag + persisted order)
const { draggedColumns, onColumnOrderChange } = useDragColumns<TableRecord>(
LOCALSTORAGE.LOGS_LIST_COLUMNS,
);
const {
orderedColumns,
orderedColumnIds,
hasSingleColumn,
handleDragEnd,
sensors,
} = useOrderedColumns({
columns,
draggedColumns,
onColumnOrderChange: onColumnOrderChange as (columns: unknown[]) => void,
});
// Column sizing (persisted). stored to localStorage.
const { columnSizing, setColumnSizing } = useColumnSizingPersistence(
orderedColumns,
);
// don't allow "remove column" when only state-indicator + one data col remain
const isAtMinimumRemovableColumns = useMemo(
() =>
orderedColumns.filter(
(column) => column.key !== 'state-indicator' && column.key !== 'expand',
).length <= 1,
[orderedColumns],
);
// Table data (TanStack row data shape)
// useTableView sends flattened log data. this would not be needed once we move to new log details view
const tableData = useMemo<TanStackTableRowData[]>(
() =>
dataSource
.map((log, rowIndex) => {
const currentLog = tableViewProps.logs[rowIndex];
if (!currentLog) {
return null;
}
return { log, currentLog, rowIndex };
})
.filter(Boolean) as TanStackTableRowData[],
[dataSource, tableViewProps.logs],
);
// TanStack columns + table instance
const tanstackColumns = useMemo<ColumnDef<TanStackTableRowData>[]>(
() =>
orderedColumns.map((column, index) => {
const isStateIndicator = column.key === 'state-indicator';
const isExpand = column.key === 'expand';
const isFixedColumn = isStateIndicator || isExpand;
const fixedWidth = isFixedColumn ? 32 : undefined;
const minWidthPx = getColumnMinWidthPx(column, orderedColumns);
const headerTitle = String(column.title || '');
return {
id: getColumnId(column),
header: headerTitle.replace(/^\w/, (character) =>
character.toUpperCase(),
),
accessorFn: (row): unknown => row.log[column.key as keyof TableRecord],
enableResizing: !isFixedColumn && index !== orderedColumns.length - 1,
minSize: fixedWidth ?? minWidthPx,
size: fixedWidth, // last column gets remaining space, so don't set initial size to avoid conflict with resizing
maxSize: fixedWidth,
cell: ({ row, getValue }): ReactElement | string | number | null => {
if (!column.render) {
return null;
}
return resolveColumnTypeRender(
column.render(
getValue(),
row.original.log,
row.original.rowIndex,
) as ColumnTypeRender<Record<string, unknown>>,
);
},
};
}),
[orderedColumns],
);
const table = useReactTable({
data: tableData,
columns: tanstackColumns,
enableColumnResizing: true,
getCoreRowModel: getCoreRowModel(),
columnResizeMode: 'onChange',
onColumnSizingChange: setColumnSizing,
state: {
columnSizing,
},
});
const tableRows = table.getRowModel().rows;
// Infinite-scroll footer UI state
const [loadMoreState, setLoadMoreState] = useState<{
active: boolean;
startCount: number;
}>({
active: false,
startCount: 0,
});
// Map to resolve full log object by id (row highlighting + indicator)
const logsById = useMemo(
() => new Map(tableViewProps.logs.map((log) => [String(log.id), log])),
[tableViewProps.logs],
);
// this is already written in parent. Check if this is needed.
useEffect(() => {
const activeLogIndex = tableViewProps.activeLogIndex ?? -1;
if (activeLogIndex < 0 || activeLogIndex >= tableRows.length) {
return;
}
virtuosoRef.current?.scrollToIndex({
index: activeLogIndex,
align: 'center',
behavior: 'auto',
});
}, [tableRows.length, tableViewProps.activeLogIndex]);
useEffect(() => {
if (!loadMoreState.active) {
return;
}
if (!isFetching || tableRows.length > loadMoreState.startCount) {
setLoadMoreState((prev) =>
prev.active ? { active: false, startCount: prev.startCount } : prev,
);
}
}, [isFetching, loadMoreState, tableRows.length]);
const handleLogCopy = useCallback(
(logId: string, event: ReactMouseEvent<HTMLElement>): void => {
event.preventDefault();
event.stopPropagation();
const urlQuery = new URLSearchParams(window.location.search);
urlQuery.delete(QueryParams.activeLogId);
urlQuery.delete(QueryParams.relativeTime);
urlQuery.set(QueryParams.activeLogId, `"${logId}"`);
const link = `${window.location.origin}${pathname}?${urlQuery.toString()}`;
setCopy(link);
toast.success('Copied to clipboard', { position: 'top-right' });
},
[pathname, setCopy],
);
const itemContent = useCallback(
(index: number): JSX.Element | null => {
const row = tableRows[index];
if (!row) {
return null;
}
return (
<TanStackRowCells
row={row}
fontSize={tableViewProps.fontSize}
onSetActiveLog={onSetActiveLog}
onClearActiveLog={onClearActiveLog}
isActiveLog={
String(activeLog?.id ?? '') === String(row.original.currentLog.id ?? '')
}
isDarkMode={isDarkMode}
onLogCopy={handleLogCopy}
isLogsExplorerPage={isLogsExplorerPage}
/>
);
},
[
activeLog?.id,
handleLogCopy,
isDarkMode,
isLogsExplorerPage,
onClearActiveLog,
onSetActiveLog,
tableRows,
tableViewProps.fontSize,
],
);
const flatHeaders = useMemo(
() => table.getFlatHeaders().filter((header) => !header.isPlaceholder),
// eslint-disable-next-line react-hooks/exhaustive-deps
[tanstackColumns],
);
const tableHeader = useCallback(() => {
const orderedColumnsById = new Map(
orderedColumns.map((column) => [getColumnId(column), column] as const),
);
return (
<DndContext
sensors={sensors}
collisionDetection={pointerWithin}
onDragEnd={handleDragEnd}
autoScroll={COLUMN_DND_AUTO_SCROLL}
>
<SortableContext
items={orderedColumnIds}
strategy={horizontalListSortingStrategy}
>
<tr>
{flatHeaders.map((header) => {
const column = orderedColumnsById.get(header.id);
if (!column) {
return null;
}
return (
<TanStackHeaderRow
key={header.id}
column={column}
header={header}
isDarkMode={isDarkMode}
fontSize={tableViewProps.fontSize}
hasSingleColumn={hasSingleColumn}
onRemoveColumn={onRemoveColumn}
canRemoveColumn={!isAtMinimumRemovableColumns}
/>
);
})}
</tr>
</SortableContext>
</DndContext>
);
}, [
flatHeaders,
handleDragEnd,
hasSingleColumn,
isDarkMode,
orderedColumnIds,
orderedColumns,
onRemoveColumn,
isAtMinimumRemovableColumns,
sensors,
tableViewProps.fontSize,
]);
const handleEndReached = useCallback(
(index: number): void => {
if (!infitiyTableProps?.onEndReached) {
return;
}
setLoadMoreState({
active: true,
startCount: tableRows.length,
});
infitiyTableProps.onEndReached(index);
},
[infitiyTableProps, tableRows.length],
);
if (isLoading) {
return <Spinner height="35px" tip="Getting Logs" />;
}
return (
<div className="tanstack-table-view-wrapper">
<TableVirtuoso
className="logs-table-virtuoso-scroll"
ref={virtuosoRef}
style={infinityDefaultStyles}
data={tableData}
totalCount={tableRows.length}
increaseViewportBy={{ top: 500, bottom: 500 }}
initialTopMostItemIndex={
tableViewProps.activeLogIndex !== -1 ? tableViewProps.activeLogIndex : 0
}
context={{ activeLog, activeContextLog, logsById }}
fixedHeaderContent={tableHeader}
itemContent={itemContent}
components={{
Table: ({ style, children }): JSX.Element => (
<TanStackTableStyled style={style}>
<colgroup>
{orderedColumns.map((column, colIndex) => {
const columnId = getColumnId(column);
const isFixedColumn =
column.key === 'expand' || column.key === 'state-indicator';
const minWidthPx = getColumnMinWidthPx(column, orderedColumns);
const persistedWidth = columnSizing[columnId];
const computedWidth = table.getColumn(columnId)?.getSize();
const effectiveWidth = persistedWidth ?? computedWidth;
if (isFixedColumn) {
return <col key={columnId} className="tanstack-fixed-col" />;
}
// Last data column should stretch to fill remaining space
const isLastColumn = colIndex === orderedColumns.length - 1;
if (isLastColumn) {
return (
<col
key={columnId}
style={{ width: '100%', minWidth: `${minWidthPx}px` }}
/>
);
}
const widthPx =
effectiveWidth != null
? Math.max(effectiveWidth, minWidthPx)
: minWidthPx;
return (
<col
key={columnId}
style={{ width: `${widthPx}px`, minWidth: `${minWidthPx}px` }}
/>
);
})}
</colgroup>
{children}
</TanStackTableStyled>
),
TableRow: TanStackCustomTableRow,
}}
{...(infitiyTableProps?.onEndReached
? { endReached: handleEndReached }
: {})}
/>
{loadMoreState.active && (
<div className="tanstack-load-more-container">
<Spinner height="20px" tip="Getting Logs" />
</div>
)}
</div>
);
},
);
export default memo(TanStackTableView);

View File

@@ -1,153 +0,0 @@
.tanstack-header-cell {
position: sticky;
top: 0;
z-index: 2;
padding: 0;
transform: translate3d(
var(--tanstack-header-translate-x, 0px),
var(--tanstack-header-translate-y, 0px),
0
);
transition: var(--tanstack-header-transition, none);
&.is-dragging {
opacity: 0.85;
}
&.is-resizing {
background: var(--l2-background-hover);
}
}
.tanstack-header-content {
display: flex;
align-items: center;
height: 100%;
min-width: 0;
width: 100%;
cursor: default;
max-width: 100%;
&.has-resize-control {
max-width: calc(100% - 5px);
}
&.has-action-control {
max-width: calc(100% - 5px);
}
&.has-resize-control.has-action-control {
max-width: calc(100% - 10px);
}
}
.tanstack-grip-slot {
display: inline-flex;
align-items: center;
justify-content: center;
width: 18px;
height: 18px;
margin-right: 4px;
flex-shrink: 0;
}
.tanstack-grip-activator {
display: inline-flex;
align-items: center;
justify-content: center;
width: 12px;
height: 12px;
cursor: grab;
color: var(--l2-foreground);
opacity: 1;
touch-action: none;
}
.tanstack-header-action-trigger {
display: inline-flex;
align-items: center;
justify-content: center;
width: 20px;
height: 20px;
cursor: pointer;
flex-shrink: 0;
color: var(--l2-foreground);
}
.tanstack-column-actions-content {
width: 140px;
padding: 0;
background: var(--l2-background);
border: 1px solid var(--l2-border);
border-radius: 4px;
box-shadow: none;
}
.tanstack-remove-column-action {
display: flex;
align-items: center;
gap: 8px;
width: 100%;
min-height: 32px;
padding: 0 8px;
border: none;
border-radius: 0;
background: transparent;
justify-content: flex-start;
cursor: pointer;
color: var(--l2-foreground);
font-size: 12px;
line-height: 16px;
font-weight: 500;
transition: background-color 120ms ease, color 120ms ease;
&:hover {
background: var(--l2-background-hover);
color: var(--l2-foreground);
.tanstack-remove-column-action-icon {
color: var(--l2-foreground);
}
}
}
.tanstack-remove-column-action-icon {
font-size: 11px;
color: var(--l2-foreground);
opacity: 0.95;
}
.tanstack-header-cell .cursor-col-resize {
position: absolute;
top: 0;
right: 0;
bottom: 0;
width: 5px;
cursor: col-resize;
z-index: 10;
touch-action: none;
background: transparent;
}
.tanstack-header-cell.is-resizing .cursor-col-resize {
background: var(--bg-robin-300);
}
.tanstack-resize-handle-line {
position: absolute;
top: 0;
bottom: 0;
left: 50%;
width: 4px;
transform: translateX(-50%);
background: var(--l2-border);
opacity: 1;
pointer-events: none;
transition: background 120ms ease, width 120ms ease;
}
.tanstack-header-cell.is-resizing .tanstack-resize-handle-line {
width: 2px;
background: var(--bg-robin-500);
transition: none;
}

View File

@@ -1,55 +0,0 @@
.tanstack-table-view-wrapper {
display: flex;
flex-direction: column;
width: 100%;
min-height: 0;
}
.tanstack-fixed-col {
width: 32px;
min-width: 32px;
max-width: 32px;
}
.tanstack-filler-col {
width: 100%;
min-width: 0;
}
.tanstack-actions-col {
width: 0;
min-width: 0;
max-width: 0;
}
.tanstack-load-more-container {
width: 100%;
min-height: 56px;
display: flex;
align-items: center;
justify-content: center;
padding: 8px 0 12px;
flex-shrink: 0;
}
.tanstack-table-virtuoso {
width: 100%;
overflow-x: scroll;
}
.tanstack-fontSize-small {
font-size: 11px;
}
.tanstack-fontSize-medium {
font-size: 13px;
}
.tanstack-fontSize-large {
font-size: 14px;
}
.tanstack-table-foot-loader-cell {
text-align: center;
padding: 8px 0;
}

View File

@@ -1,29 +0,0 @@
import { ColumnSizingState } from '@tanstack/react-table';
import { ColumnTypeRender } from 'components/Logs/TableView/types';
import { ILog } from 'types/api/logs/log';
export type TableRecord = Record<string, unknown>;
export type LogsTableColumnDef = {
key?: string | number;
title?: string;
render?: (
value: unknown,
record: TableRecord,
index: number,
) => ColumnTypeRender<Record<string, unknown>>;
};
export type OrderedColumn = LogsTableColumnDef & {
key: string | number;
};
export type TanStackTableRowData = {
log: TableRecord;
currentLog: ILog;
rowIndex: number;
};
export type PersistedColumnSizing = {
sizing: ColumnSizingState;
};

View File

@@ -1,111 +0,0 @@
import { Dispatch, SetStateAction, useEffect, useMemo, useState } from 'react';
import { ColumnSizingState } from '@tanstack/react-table';
import getFromLocalstorage from 'api/browser/localstorage/get';
import setToLocalstorage from 'api/browser/localstorage/set';
import { LOCALSTORAGE } from 'constants/localStorage';
import { OrderedColumn, PersistedColumnSizing } from './types';
import { getColumnId } from './utils';
const COLUMN_SIZING_PERSIST_DEBOUNCE_MS = 250;
const sanitizeSizing = (input: unknown): ColumnSizingState => {
if (!input || typeof input !== 'object') {
return {};
}
return Object.entries(
input as Record<string, unknown>,
).reduce<ColumnSizingState>((acc, [key, value]) => {
if (typeof value !== 'number' || !Number.isFinite(value) || value <= 0) {
return acc;
}
acc[key] = value;
return acc;
}, {});
};
const readPersistedColumnSizing = (): ColumnSizingState => {
const rawSizing = getFromLocalstorage(LOCALSTORAGE.LOGS_LIST_COLUMN_SIZING);
if (!rawSizing) {
return {};
}
try {
const parsed = JSON.parse(rawSizing) as
| PersistedColumnSizing
| ColumnSizingState;
const sizing = ('sizing' in parsed
? parsed.sizing
: parsed) as ColumnSizingState;
return sanitizeSizing(sizing);
} catch (error) {
console.error('Failed to parse persisted log column sizing', error);
return {};
}
};
type UseColumnSizingPersistenceResult = {
columnSizing: ColumnSizingState;
setColumnSizing: Dispatch<SetStateAction<ColumnSizingState>>;
};
export const useColumnSizingPersistence = (
orderedColumns: OrderedColumn[],
): UseColumnSizingPersistenceResult => {
const [columnSizing, setColumnSizing] = useState<ColumnSizingState>(() =>
readPersistedColumnSizing(),
);
const orderedColumnIds = useMemo(
() => orderedColumns.map((column) => getColumnId(column)),
[orderedColumns],
);
useEffect(() => {
if (orderedColumnIds.length === 0) {
return;
}
const validColumnIds = new Set(orderedColumnIds);
const nonResizableColumnIds = new Set(
orderedColumns
.filter(
(column) => column.key === 'expand' || column.key === 'state-indicator',
)
.map((column) => getColumnId(column)),
);
setColumnSizing((previousSizing) => {
const nextSizing = Object.entries(previousSizing).reduce<ColumnSizingState>(
(acc, [columnId, size]) => {
if (!validColumnIds.has(columnId) || nonResizableColumnIds.has(columnId)) {
return acc;
}
acc[columnId] = size;
return acc;
},
{},
);
const hasChanged =
Object.keys(nextSizing).length !== Object.keys(previousSizing).length ||
Object.entries(nextSizing).some(
([columnId, size]) => previousSizing[columnId] !== size,
);
return hasChanged ? nextSizing : previousSizing;
});
}, [orderedColumnIds, orderedColumns]);
useEffect(() => {
const timeoutId = window.setTimeout(() => {
const persistedSizing = { sizing: columnSizing };
setToLocalstorage(
LOCALSTORAGE.LOGS_LIST_COLUMN_SIZING,
JSON.stringify(persistedSizing),
);
}, COLUMN_SIZING_PERSIST_DEBOUNCE_MS);
return (): void => window.clearTimeout(timeoutId);
}, [columnSizing]);
return { columnSizing, setColumnSizing };
};

View File

@@ -1,108 +0,0 @@
import { useCallback, useMemo } from 'react';
import {
DragEndEvent,
PointerSensor,
useSensor,
useSensors,
} from '@dnd-kit/core';
import { arrayMove } from '@dnd-kit/sortable';
import { getDraggedColumns } from 'hooks/useDragColumns/utils';
import { OrderedColumn, TableRecord } from './types';
import { getColumnId } from './utils';
type UseOrderedColumnsProps = {
columns: unknown[];
draggedColumns: unknown[];
onColumnOrderChange: (columns: unknown[]) => void;
};
type UseOrderedColumnsResult = {
orderedColumns: OrderedColumn[];
orderedColumnIds: string[];
hasSingleColumn: boolean;
handleDragEnd: (event: DragEndEvent) => void;
sensors: ReturnType<typeof useSensors>;
};
export const useOrderedColumns = ({
columns,
draggedColumns,
onColumnOrderChange,
}: UseOrderedColumnsProps): UseOrderedColumnsResult => {
const baseColumns = useMemo<OrderedColumn[]>(
() =>
getDraggedColumns<TableRecord>(
columns as never[],
draggedColumns as never[],
).filter(
(column): column is OrderedColumn =>
typeof column.key === 'string' || typeof column.key === 'number',
),
[columns, draggedColumns],
);
const orderedColumns = useMemo(() => {
const stateIndicatorIndex = baseColumns.findIndex(
(column) => column.key === 'state-indicator',
);
if (stateIndicatorIndex <= 0) {
return baseColumns;
}
const pinned = baseColumns[stateIndicatorIndex];
const rest = baseColumns.filter((_, i) => i !== stateIndicatorIndex);
return [pinned, ...rest];
}, [baseColumns]);
const handleDragEnd = useCallback(
(event: DragEndEvent): void => {
const { active, over } = event;
if (!over || active.id === over.id) {
return;
}
// Don't allow moving the state-indicator column
if (String(active.id) === 'state-indicator') {
return;
}
const oldIndex = orderedColumns.findIndex(
(column) => getColumnId(column) === String(active.id),
);
const newIndex = orderedColumns.findIndex(
(column) => getColumnId(column) === String(over.id),
);
if (oldIndex === -1 || newIndex === -1) {
return;
}
const nextColumns = arrayMove(orderedColumns, oldIndex, newIndex);
onColumnOrderChange(nextColumns as unknown[]);
},
[onColumnOrderChange, orderedColumns],
);
const orderedColumnIds = useMemo(
() => orderedColumns.map((column) => getColumnId(column)),
[orderedColumns],
);
const hasSingleColumn = useMemo(
() =>
orderedColumns.filter((column) => column.key !== 'state-indicator')
.length === 1,
[orderedColumns],
);
const sensors = useSensors(
useSensor(PointerSensor, {
activationConstraint: { distance: 4 },
}),
);
return {
orderedColumns,
orderedColumnIds,
hasSingleColumn,
handleDragEnd,
sensors,
};
};

View File

@@ -1,61 +0,0 @@
import { cloneElement, isValidElement, ReactElement } from 'react';
import { ColumnTypeRender } from 'components/Logs/TableView/types';
import { OrderedColumn } from './types';
export const getColumnId = (column: OrderedColumn): string =>
String(column.key);
/** Browser default root font size; TanStack column sizing uses px. */
const REM_PX = 16;
const MIN_WIDTH_OTHER_REM = 12;
const MIN_WIDTH_BODY_REM = 40;
/** When total column count is below this, body column min width is doubled (more horizontal space for few columns). */
export const FEW_COLUMNS_BODY_MIN_WIDTH_THRESHOLD = 4;
/**
* Minimum width (px) for TanStack column defs + colgroup.
* Design: state/expand 32px; body min 40rem (doubled when fewer than
* {@link FEW_COLUMNS_BODY_MIN_WIDTH_THRESHOLD} total columns); other columns use rem→px (16px root).
*/
export const getColumnMinWidthPx = (
column: OrderedColumn,
orderedColumns?: OrderedColumn[],
): number => {
const key = String(column.key);
if (key === 'state-indicator' || key === 'expand') {
return 32;
}
if (key === 'body') {
const base = MIN_WIDTH_BODY_REM * REM_PX;
const fewColumns =
orderedColumns != null &&
orderedColumns.length < FEW_COLUMNS_BODY_MIN_WIDTH_THRESHOLD;
return fewColumns ? base * 1.5 : base;
}
return MIN_WIDTH_OTHER_REM * REM_PX;
};
export const resolveColumnTypeRender = (
rendered: ColumnTypeRender<Record<string, unknown>>,
): ReactElement | string | number | null => {
if (
rendered &&
typeof rendered === 'object' &&
'children' in rendered &&
isValidElement(rendered.children)
) {
const { children, props } = rendered as {
children: ReactElement;
props?: Record<string, unknown>;
};
return cloneElement(children, props || {});
}
if (rendered && typeof rendered === 'object' && isValidElement(rendered)) {
return rendered;
}
return typeof rendered === 'string' || typeof rendered === 'number'
? rendered
: null;
};

View File

@@ -25,9 +25,9 @@ import { ILog } from 'types/api/logs/log';
import { DataSource, StringOperators } from 'types/common/queryBuilder';
import NoLogs from '../NoLogs/NoLogs';
import InfinityTableView from './InfinityTableView';
import { LogsExplorerListProps } from './LogsExplorerList.interfaces';
import { InfinityWrapperStyled } from './styles';
import TanStackTableView from './TanStackTableView';
import {
convertKeysToColumnFields,
getEmptyLogsListConfig,
@@ -61,7 +61,7 @@ function LogsExplorerList({
handleCloseLogDetail,
} = useLogDetailHandlers();
const { options, config } = useOptionsMenu({
const { options } = useOptionsMenu({
storageKey: LOCALSTORAGE.LOGS_LIST_OPTIONS,
dataSource: DataSource.LOGS,
aggregateOperator:
@@ -155,10 +155,9 @@ function LogsExplorerList({
if (options.format === 'table') {
return (
<TanStackTableView
<InfinityTableView
ref={ref}
isLoading={isLoading}
isFetching={isFetching}
tableViewProps={{
logs,
fields: selectedFields,
@@ -173,7 +172,6 @@ function LogsExplorerList({
onSetActiveLog={handleSetActiveLog}
onClearActiveLog={handleCloseLogDetail}
activeLog={activeLog}
onRemoveColumn={config.addColumn?.onRemove}
/>
);
}
@@ -218,13 +216,11 @@ function LogsExplorerList({
logs,
onEndReached,
getItemContent,
isFetching,
selectedFields,
handleChangeSelectedView,
handleSetActiveLog,
handleCloseLogDetail,
activeLog,
config.addColumn?.onRemove,
]);
const isTraceToLogsNavigation = useMemo(() => {

View File

@@ -1,38 +0,0 @@
.logs-table-virtuoso-scroll {
scrollbar-width: thin;
scrollbar-color: var(--bg-slate-300) transparent;
&::-webkit-scrollbar {
width: 4px;
height: 4px;
}
&::-webkit-scrollbar-corner {
background: transparent;
}
&::-webkit-scrollbar-track {
background: transparent;
}
&::-webkit-scrollbar-thumb {
background: var(--bg-slate-300);
border-radius: 9999px;
}
&::-webkit-scrollbar-thumb:hover {
background: var(--bg-slate-200);
}
}
.lightMode .logs-table-virtuoso-scroll {
scrollbar-color: var(--bg-vanilla-300) transparent;
&::-webkit-scrollbar-thumb {
background: var(--bg-vanilla-300);
}
&::-webkit-scrollbar-thumb:hover {
background: var(--bg-vanilla-100);
}
}

View File

@@ -2,7 +2,7 @@ import styled from 'styled-components';
export const InfinityWrapperStyled = styled.div`
flex: 1;
height: 40rem !important;
display: flex;
height: 100%;
min-height: 0;
`;

View File

@@ -1,7 +1,6 @@
.logs-explorer-views-container {
margin-bottom: 24px;
flex: 1;
min-height: 0;
display: flex;
flex-direction: column;
@@ -10,7 +9,6 @@
display: flex;
flex-direction: column;
flex: 1;
min-height: 0;
padding-bottom: 10px;
.views-tabs-container {
@@ -197,7 +195,6 @@
.logs-explorer-views-type-content {
flex: 1;
min-height: 0;
display: flex;
flex-direction: column;
@@ -213,32 +210,12 @@
}
}
.table-view-container {
flex: 1;
min-height: 0;
overflow-y: visible;
}
.time-series-view-container {
flex: 1;
display: flex;
flex-direction: column;
min-height: 0;
overflow-y: visible;
.time-series-view-container-header {
display: flex;
justify-content: flex-start;
align-items: center;
padding: 12px;
flex-shrink: 0;
}
.time-series-view {
flex-shrink: 0;
height: 65vh;
min-height: 450px;
padding-bottom: 140px;
}
}
}

View File

@@ -499,18 +499,16 @@ function LogsExplorerViewsContainer({
</div>
)}
{selectedPanelType === PANEL_TYPES.TABLE && !showLiveLogs && (
<div className="table-view-container">
<LogsExplorerTable
data={
(data?.payload?.data?.newResult?.data?.result ||
data?.payload?.data?.result ||
[]) as QueryDataV3[]
}
isLoading={isLoading || isFetching}
isError={isError}
error={error as APIError}
/>
</div>
<LogsExplorerTable
data={
(data?.payload?.data?.newResult?.data?.result ||
data?.payload?.data?.result ||
[]) as QueryDataV3[]
}
isLoading={isLoading || isFetching}
isError={isError}
error={error as APIError}
/>
)}
</div>
</div>

View File

@@ -14,10 +14,6 @@ import { QueryParams } from 'constants/query';
import ROUTES from 'constants/routes';
import NewExplorerCTA from 'container/NewExplorerCTA';
import dayjs, { Dayjs } from 'dayjs';
import {
useGlobalTimeQueryInvalidate,
useIsGlobalTimeQueryRefreshing,
} from 'hooks/globalTime';
import { useQueryBuilder } from 'hooks/queryBuilder/useQueryBuilder';
import { useSafeNavigate } from 'hooks/useSafeNavigate';
import useUrlQuery from 'hooks/useUrlQuery';
@@ -356,10 +352,7 @@ function DateTimeSelection({
],
);
const isRefreshingQueries = useIsGlobalTimeQueryRefreshing();
const invalidateQueries = useGlobalTimeQueryInvalidate();
const onRefreshHandler = (): void => {
invalidateQueries();
onSelectHandler(selectedTime);
onLastRefreshHandler();
};
@@ -739,11 +732,7 @@ function DateTimeSelection({
{showAutoRefresh && selectedTime !== 'custom' && (
<div className="refresh-actions">
<FormItem hidden={refreshButtonHidden} className="refresh-btn">
<Button
icon={<SyncOutlined />}
loading={!!isRefreshingQueries}
onClick={onRefreshHandler}
/>
<Button icon={<SyncOutlined />} onClick={onRefreshHandler} />
</FormItem>
<FormItem>

View File

@@ -1,2 +0,0 @@
export { useGlobalTimeQueryInvalidate } from './useGlobalTimeQueryInvalidate';
export { useIsGlobalTimeQueryRefreshing } from './useIsGlobalTimeQueryRefreshing';

View File

@@ -1,16 +0,0 @@
import { useCallback } from 'react';
import { useQueryClient } from 'react-query';
import { REACT_QUERY_KEY } from 'constants/reactQueryKeys';
/**
* Use when you want to invalida any query tracked by {@link REACT_QUERY_KEY.AUTO_REFRESH_QUERY}
*/
export function useGlobalTimeQueryInvalidate(): () => Promise<void> {
const queryClient = useQueryClient();
return useCallback(async () => {
return await queryClient.invalidateQueries({
queryKey: [REACT_QUERY_KEY.AUTO_REFRESH_QUERY],
});
}, [queryClient]);
}

View File

@@ -1,13 +0,0 @@
import { useIsFetching } from 'react-query';
import { REACT_QUERY_KEY } from 'constants/reactQueryKeys';
/**
* Use when you want to know if any query tracked by {@link REACT_QUERY_KEY.AUTO_REFRESH_QUERY} is refreshing
*/
export function useIsGlobalTimeQueryRefreshing(): boolean {
return (
useIsFetching({
queryKey: [REACT_QUERY_KEY.AUTO_REFRESH_QUERY],
}) > 0
);
}

View File

@@ -5,7 +5,6 @@ import { QueryClient, QueryClientProvider } from 'react-query';
import { Provider } from 'react-redux';
import AppRoutes from 'AppRoutes';
import { AxiosError } from 'axios';
import { GlobalTimeStoreAdapter } from 'components/GlobalTimeStoreAdapter/GlobalTimeStoreAdapter';
import { ThemeProvider } from 'hooks/useDarkMode';
import { NuqsAdapter } from 'nuqs/adapters/react';
import { AppProvider } from 'providers/App/App';
@@ -52,7 +51,6 @@ if (container) {
<TimezoneProvider>
<QueryClientProvider client={queryClient}>
<Provider store={store}>
<GlobalTimeStoreAdapter />
<AppProvider>
<AppRoutes />
</AppProvider>

View File

@@ -1,9 +1,6 @@
.logs-module-page {
display: flex;
height: 100%;
min-height: 0;
overflow: hidden;
.log-quick-filter-left-section {
width: 0%;
flex-shrink: 0;
@@ -13,19 +10,13 @@
display: flex;
flex-direction: column;
width: 100%;
min-height: 0;
.log-explorer-query-container {
display: flex;
flex-direction: column;
flex: 1;
min-height: 0;
overflow-y: auto;
overflow-x: hidden;
.logs-explorer-views {
flex: 1;
min-height: 0;
display: flex;
flex-direction: column;
}
@@ -35,18 +26,6 @@
&.filter-visible {
.log-quick-filter-left-section {
width: 260px;
height: 100%;
overflow: visible;
min-height: 0;
position: relative;
z-index: 2;
display: flex;
flex-direction: column;
.quick-filters-container {
flex: 1;
min-height: 0;
}
}
.log-module-right-section {

View File

@@ -1,14 +1,10 @@
.logs-module-container {
flex: 1;
min-height: 0;
display: flex;
flex-direction: column;
.ant-tabs {
flex: 1;
min-height: 0;
display: flex;
flex-direction: column;
}
.ant-tabs-nav {
@@ -22,17 +18,14 @@
.ant-tabs-content-holder {
display: flex;
min-height: 0;
.ant-tabs-content {
flex: 1;
min-height: 0;
display: flex;
flex-direction: column;
.ant-tabs-tabpane {
flex: 1;
min-height: 0;
display: flex;
flex-direction: column;
}

View File

@@ -1,204 +0,0 @@
import { act, renderHook } from '@testing-library/react';
import { DEFAULT_TIME_RANGE } from 'container/TopNav/DateTimeSelectionV2/constants';
import { useGlobalTimeStore } from '../globalTimeStore';
import { GlobalTimeSelectedTime } from '../types';
import { createCustomTimeRange, NANO_SECOND_MULTIPLIER } from '../utils';
describe('globalTimeStore', () => {
beforeEach(() => {
const { result } = renderHook(() => useGlobalTimeStore());
act(() => {
result.current.setSelectedTime(DEFAULT_TIME_RANGE, 0);
});
});
describe('initial state', () => {
it(`should have default selectedTime of ${DEFAULT_TIME_RANGE}`, () => {
const { result } = renderHook(() => useGlobalTimeStore());
expect(result.current.selectedTime).toBe(DEFAULT_TIME_RANGE);
});
it('should have isRefreshEnabled as false by default', () => {
const { result } = renderHook(() => useGlobalTimeStore());
expect(result.current.isRefreshEnabled).toBe(false);
});
it('should have refreshInterval as 0 by default', () => {
const { result } = renderHook(() => useGlobalTimeStore());
expect(result.current.refreshInterval).toBe(0);
});
});
describe('setSelectedTime', () => {
it('should update selectedTime', () => {
const { result } = renderHook(() => useGlobalTimeStore());
act(() => {
result.current.setSelectedTime('15m');
});
expect(result.current.selectedTime).toBe('15m');
});
it('should update refreshInterval when provided', () => {
const { result } = renderHook(() => useGlobalTimeStore());
act(() => {
result.current.setSelectedTime('15m', 5000);
});
expect(result.current.refreshInterval).toBe(5000);
});
it('should keep existing refreshInterval when not provided', () => {
const { result } = renderHook(() => useGlobalTimeStore());
act(() => {
result.current.setSelectedTime('15m', 5000);
});
act(() => {
result.current.setSelectedTime('1h');
});
expect(result.current.refreshInterval).toBe(5000);
});
it('should enable refresh for relative time with refreshInterval > 0', () => {
const { result } = renderHook(() => useGlobalTimeStore());
act(() => {
result.current.setSelectedTime('15m', 5000);
});
expect(result.current.isRefreshEnabled).toBe(true);
});
it('should disable refresh for relative time with refreshInterval = 0', () => {
const { result } = renderHook(() => useGlobalTimeStore());
act(() => {
result.current.setSelectedTime('15m', 0);
});
expect(result.current.isRefreshEnabled).toBe(false);
});
it('should disable refresh for custom time range even with refreshInterval > 0', () => {
const { result } = renderHook(() => useGlobalTimeStore());
const customTime = createCustomTimeRange(1000000000, 2000000000);
act(() => {
result.current.setSelectedTime(customTime, 5000);
});
expect(result.current.isRefreshEnabled).toBe(false);
expect(result.current.refreshInterval).toBe(5000);
});
it('should handle various relative time formats', () => {
const { result } = renderHook(() => useGlobalTimeStore());
const timeFormats: GlobalTimeSelectedTime[] = [
'1m',
'5m',
'15m',
'30m',
'1h',
'3h',
'6h',
'1d',
'1w',
];
timeFormats.forEach((time) => {
act(() => {
result.current.setSelectedTime(time, 10000);
});
expect(result.current.selectedTime).toBe(time);
expect(result.current.isRefreshEnabled).toBe(true);
});
});
});
describe('getMinMaxTime', () => {
beforeEach(() => {
jest.useFakeTimers();
jest.setSystemTime(new Date('2024-01-15T12:00:00.000Z'));
});
afterEach(() => {
jest.useRealTimers();
});
it('should return min/max time for custom time range', () => {
const { result } = renderHook(() => useGlobalTimeStore());
const minTime = 1000000000;
const maxTime = 2000000000;
const customTime = createCustomTimeRange(minTime, maxTime);
act(() => {
result.current.setSelectedTime(customTime);
});
const {
minTime: resultMin,
maxTime: resultMax,
} = result.current.getMinMaxTime();
expect(resultMin).toBe(minTime);
expect(resultMax).toBe(maxTime);
});
it('should compute fresh min/max time for relative time', () => {
const { result } = renderHook(() => useGlobalTimeStore());
act(() => {
result.current.setSelectedTime('15m');
});
const { minTime, maxTime } = result.current.getMinMaxTime();
const now = Date.now() * NANO_SECOND_MULTIPLIER;
const fifteenMinutesNs = 15 * 60 * 1000 * NANO_SECOND_MULTIPLIER;
expect(maxTime).toBe(now);
expect(minTime).toBe(now - fifteenMinutesNs);
});
it('should return different values on subsequent calls for relative time', () => {
const { result } = renderHook(() => useGlobalTimeStore());
act(() => {
result.current.setSelectedTime('15m');
});
const first = result.current.getMinMaxTime();
// Advance time by 1 second
act(() => {
jest.advanceTimersByTime(1000);
});
const second = result.current.getMinMaxTime();
// maxTime should be different (1 second later)
expect(second.maxTime).toBe(first.maxTime + 1000 * NANO_SECOND_MULTIPLIER);
expect(second.minTime).toBe(first.minTime + 1000 * NANO_SECOND_MULTIPLIER);
});
});
describe('store isolation', () => {
it('should share state between multiple hook instances', () => {
const { result: result1 } = renderHook(() => useGlobalTimeStore());
const { result: result2 } = renderHook(() => useGlobalTimeStore());
act(() => {
result1.current.setSelectedTime('1h', 10000);
});
expect(result2.current.selectedTime).toBe('1h');
expect(result2.current.refreshInterval).toBe(10000);
expect(result2.current.isRefreshEnabled).toBe(true);
});
});
});

View File

@@ -1,139 +0,0 @@
import {
createCustomTimeRange,
CUSTOM_TIME_SEPARATOR,
isCustomTimeRange,
NANO_SECOND_MULTIPLIER,
parseCustomTimeRange,
parseSelectedTime,
} from '../utils';
describe('globalTime/utils', () => {
describe('CUSTOM_TIME_SEPARATOR', () => {
it('should be defined as ||_||', () => {
expect(CUSTOM_TIME_SEPARATOR).toBe('||_||');
});
});
describe('isCustomTimeRange', () => {
it('should return true for custom time range strings', () => {
expect(isCustomTimeRange('1000000000||_||2000000000')).toBe(true);
expect(isCustomTimeRange('0||_||0')).toBe(true);
});
it('should return false for relative time strings', () => {
expect(isCustomTimeRange('15m')).toBe(false);
expect(isCustomTimeRange('1h')).toBe(false);
expect(isCustomTimeRange('1d')).toBe(false);
expect(isCustomTimeRange('30s')).toBe(false);
});
it('should return false for empty string', () => {
expect(isCustomTimeRange('')).toBe(false);
});
});
describe('createCustomTimeRange', () => {
it('should create a custom time range string from min and max times', () => {
const minTime = 1000000000;
const maxTime = 2000000000;
const result = createCustomTimeRange(minTime, maxTime);
expect(result).toBe(`${minTime}${CUSTOM_TIME_SEPARATOR}${maxTime}`);
});
it('should handle zero values', () => {
const result = createCustomTimeRange(0, 0);
expect(result).toBe(`0${CUSTOM_TIME_SEPARATOR}0`);
});
it('should handle large nanosecond timestamps', () => {
const minTime = 1700000000000000000;
const maxTime = 1700000001000000000;
const result = createCustomTimeRange(minTime, maxTime);
expect(result).toBe(`${minTime}${CUSTOM_TIME_SEPARATOR}${maxTime}`);
});
});
describe('parseCustomTimeRange', () => {
it('should parse a valid custom time range string', () => {
const minTime = 1000000000;
const maxTime = 2000000000;
const timeString = `${minTime}${CUSTOM_TIME_SEPARATOR}${maxTime}`;
const result = parseCustomTimeRange(timeString);
expect(result).toEqual({ minTime, maxTime });
});
it('should return null for non-custom time range strings', () => {
expect(parseCustomTimeRange('15m')).toBeNull();
expect(parseCustomTimeRange('1h')).toBeNull();
});
it('should return null for invalid numeric values', () => {
expect(parseCustomTimeRange(`abc${CUSTOM_TIME_SEPARATOR}def`)).toBeNull();
expect(parseCustomTimeRange(`123${CUSTOM_TIME_SEPARATOR}def`)).toBeNull();
expect(parseCustomTimeRange(`abc${CUSTOM_TIME_SEPARATOR}456`)).toBeNull();
});
it('should handle zero values', () => {
const result = parseCustomTimeRange(`0${CUSTOM_TIME_SEPARATOR}0`);
expect(result).toEqual({ minTime: 0, maxTime: 0 });
});
});
describe('parseSelectedTime', () => {
beforeEach(() => {
jest.useFakeTimers();
jest.setSystemTime(new Date('2024-01-15T12:00:00.000Z'));
});
afterEach(() => {
jest.useRealTimers();
});
it('should parse custom time range and return min/max values', () => {
const minTime = 1000000000;
const maxTime = 2000000000;
const timeString = createCustomTimeRange(minTime, maxTime);
const result = parseSelectedTime(timeString);
expect(result).toEqual({ minTime, maxTime });
});
it('should return fallback for invalid custom time range', () => {
const invalidCustom = `invalid${CUSTOM_TIME_SEPARATOR}values`;
const result = parseSelectedTime(invalidCustom);
const now = Date.now() * NANO_SECOND_MULTIPLIER;
const fallbackDuration = 30 * 1000 * NANO_SECOND_MULTIPLIER; // 30s in nanoseconds
expect(result.maxTime).toBe(now);
expect(result.minTime).toBe(now - fallbackDuration);
});
it('should parse relative time strings using getMinMaxForSelectedTime', () => {
const result = parseSelectedTime('15m');
const now = Date.now() * NANO_SECOND_MULTIPLIER;
// 15 minutes in nanoseconds
const fifteenMinutesNs = 15 * 60 * 1000 * NANO_SECOND_MULTIPLIER;
expect(result.maxTime).toBe(now);
expect(result.minTime).toBe(now - fifteenMinutesNs);
});
it('should parse 1h relative time', () => {
const result = parseSelectedTime('1h');
const now = Date.now() * NANO_SECOND_MULTIPLIER;
// 1 hour in nanoseconds
const oneHourNs = 60 * 60 * 1000 * NANO_SECOND_MULTIPLIER;
expect(result.maxTime).toBe(now);
expect(result.minTime).toBe(now - oneHourNs);
});
it('should parse 1d relative time', () => {
const result = parseSelectedTime('1d');
const now = Date.now() * NANO_SECOND_MULTIPLIER;
// 1 day in nanoseconds
const oneDayNs = 24 * 60 * 60 * 1000 * NANO_SECOND_MULTIPLIER;
expect(result.maxTime).toBe(now);
expect(result.minTime).toBe(now - oneDayNs);
});
});
});

View File

@@ -1,33 +0,0 @@
import { DEFAULT_TIME_RANGE } from 'container/TopNav/DateTimeSelectionV2/constants';
import { create } from 'zustand';
import {
IGlobalTimeStoreActions,
IGlobalTimeStoreState,
ParsedTimeRange,
} from './types';
import { isCustomTimeRange, parseSelectedTime } from './utils';
export type IGlobalTimeStore = IGlobalTimeStoreState & IGlobalTimeStoreActions;
export const useGlobalTimeStore = create<IGlobalTimeStore>((set, get) => ({
selectedTime: DEFAULT_TIME_RANGE,
isRefreshEnabled: false,
refreshInterval: 0,
setSelectedTime: (selectedTime, refreshInterval): void => {
set((state) => {
const newRefreshInterval = refreshInterval ?? state.refreshInterval;
const isCustom = isCustomTimeRange(selectedTime);
return {
selectedTime,
refreshInterval: newRefreshInterval,
isRefreshEnabled: !isCustom && newRefreshInterval > 0,
};
});
},
getMinMaxTime: (): ParsedTimeRange => {
const { selectedTime } = get();
return parseSelectedTime(selectedTime);
},
}));

View File

@@ -1,9 +0,0 @@
export { useGlobalTimeStore } from './globalTimeStore';
export type { IGlobalTimeStoreState, ParsedTimeRange } from './types';
export {
createCustomTimeRange,
CUSTOM_TIME_SEPARATOR,
isCustomTimeRange,
parseCustomTimeRange,
parseSelectedTime,
} from './utils';

View File

@@ -1,52 +0,0 @@
import { Time } from 'container/TopNav/DateTimeSelectionV2/types';
export type CustomTimeRangeSeparator = '||_||';
export type CustomTimeRange = `${number}${CustomTimeRangeSeparator}${number}`;
export type GlobalTimeSelectedTime = Time | CustomTimeRange;
export interface IGlobalTimeStoreState {
/**
* The selected time range, can be:
* - Relative duration: '1m', '5m', '15m', '1h', '1d', etc.
* - Custom range: '<minTimeUnixNano>||_||<maxTimeUnixNano>' format
*/
selectedTime: GlobalTimeSelectedTime;
/**
* Whether auto-refresh is enabled.
* Automatically computed: true for duration-based times, false for custom ranges.
*/
isRefreshEnabled: boolean;
/**
* The refresh interval in milliseconds (e.g., 5000 for 5s, 30000 for 30s)
* Only used when isRefreshEnabled is true
*/
refreshInterval: number;
}
export interface ParsedTimeRange {
minTime: number;
maxTime: number;
}
export interface IGlobalTimeStoreActions {
/**
* Set the selected time and optionally the refresh interval.
* isRefreshEnabled is automatically computed:
* - Custom time ranges: always false
* - Duration times with refreshInterval > 0: true
* - Duration times with refreshInterval = 0: false
*/
setSelectedTime: (
selectedTime: GlobalTimeSelectedTime,
refreshInterval?: number,
) => void;
/**
* Get the current min/max time values parsed from selectedTime.
* For durations, computes fresh values based on Date.now().
* For custom ranges, extracts the stored values.
*/
getMinMaxTime: () => ParsedTimeRange;
}

View File

@@ -1,87 +0,0 @@
import { Time } from 'container/TopNav/DateTimeSelectionV2/types';
import { getMinMaxForSelectedTime } from 'lib/getMinMax';
import { REACT_QUERY_KEY } from '../../constants/reactQueryKeys';
import {
CustomTimeRange,
CustomTimeRangeSeparator,
GlobalTimeSelectedTime,
ParsedTimeRange,
} from './types';
/**
* Custom time range separator used in the selectedTime string
*/
export const CUSTOM_TIME_SEPARATOR: CustomTimeRangeSeparator = '||_||';
/**
* Check if selectedTime represents a custom time range
*/
export function isCustomTimeRange(selectedTime: string): boolean {
return selectedTime.includes(CUSTOM_TIME_SEPARATOR);
}
/**
* Create a custom time range string from min/max times (in nanoseconds)
*/
export function createCustomTimeRange(
minTime: number,
maxTime: number,
): CustomTimeRange {
return `${minTime}${CUSTOM_TIME_SEPARATOR}${maxTime}`;
}
/**
* Parse the custom time range string to get min/max times (in nanoseconds)
*/
export function parseCustomTimeRange(
selectedTime: string,
): ParsedTimeRange | null {
if (!isCustomTimeRange(selectedTime)) {
return null;
}
const [minStr, maxStr] = selectedTime.split(CUSTOM_TIME_SEPARATOR);
const minTime = parseInt(minStr, 10);
const maxTime = parseInt(maxStr, 10);
if (Number.isNaN(minTime) || Number.isNaN(maxTime)) {
return null;
}
return { minTime, maxTime };
}
export const NANO_SECOND_MULTIPLIER = 1000000;
const fallbackDurationInNanoSeconds = 30 * 1000 * NANO_SECOND_MULTIPLIER; // 30s
/**
* Parse the selectedTime string to get min/max time values.
* For relative times, computes fresh values based on Date.now().
* For custom times, extracts the stored min/max values.
*/
export function parseSelectedTime(selectedTime: string): ParsedTimeRange {
if (isCustomTimeRange(selectedTime)) {
const parsed = parseCustomTimeRange(selectedTime);
if (parsed) {
return parsed;
}
// Fallback to current time if parsing fails
const now = Date.now() * NANO_SECOND_MULTIPLIER;
return { minTime: now - fallbackDurationInNanoSeconds, maxTime: now };
}
// It's a relative time like '15m', '1h', etc.
// Use getMinMaxForSelectedTime which computes from Date.now()
return getMinMaxForSelectedTime(selectedTime as Time, 0, 0);
}
/**
* Use to build your react-query key for auto-refresh queries
*/
export function getAutoRefreshQueryKey(
selectedTime: GlobalTimeSelectedTime,
...queryParts: unknown[]
): unknown[] {
return [REACT_QUERY_KEY.AUTO_REFRESH_QUERY, ...queryParts, selectedTime];
}

View File

@@ -6017,19 +6017,26 @@
dependencies:
defer-to-connect "^2.0.0"
"@tanstack/react-table@8.21.3", "@tanstack/react-table@^8.21.3":
"@tanstack/react-table@8.20.6":
version "8.20.6"
resolved "https://registry.yarnpkg.com/@tanstack/react-table/-/react-table-8.20.6.tgz#a1f3103327aa59aa621931f4087a7604a21054d0"
integrity sha512-w0jluT718MrOKthRcr2xsjqzx+oEM7B7s/XXyfs19ll++hlId3fjTm+B2zrR3ijpANpkzBAr15j1XGVOMxpggQ==
dependencies:
"@tanstack/table-core" "8.20.5"
"@tanstack/react-table@^8.21.3":
version "8.21.3"
resolved "https://registry.yarnpkg.com/@tanstack/react-table/-/react-table-8.21.3.tgz#2c38c747a5731c1a07174fda764b9c2b1fb5e91b"
integrity sha512-5nNMTSETP4ykGegmVkhjcS8tTLW6Vl4axfEGQN3v0zdHYbK4UfoqfPChclTrJ4EoK9QynqAu9oUf8VEmrpZ5Ww==
dependencies:
"@tanstack/table-core" "8.21.3"
"@tanstack/react-virtual@3.13.22":
version "3.13.22"
resolved "https://registry.yarnpkg.com/@tanstack/react-virtual/-/react-virtual-3.13.22.tgz#9a5529dee4010f33272ae3b3e3728dee317b3b42"
integrity sha512-EaOrBBJLi3M0bTMQRjGkxLXRw7Gizwntoy5E2Q2UnSbML7Mo2a1P/Hfkw5tw9FLzK62bj34Jl6VNbQfRV6eJcA==
"@tanstack/react-virtual@3.11.2":
version "3.11.2"
resolved "https://registry.yarnpkg.com/@tanstack/react-virtual/-/react-virtual-3.11.2.tgz#d6b9bd999c181f0a2edce270c87a2febead04322"
integrity sha512-OuFzMXPF4+xZgx8UzJha0AieuMihhhaWG0tCqpp6tDzlFwOmNBPYMuLOtMJ1Tr4pXLHmgjcWhG6RlknY2oNTdQ==
dependencies:
"@tanstack/virtual-core" "3.13.22"
"@tanstack/virtual-core" "3.11.2"
"@tanstack/react-virtual@^3.13.9":
version "3.13.12"
@@ -6038,21 +6045,26 @@
dependencies:
"@tanstack/virtual-core" "3.13.12"
"@tanstack/table-core@8.20.5":
version "8.20.5"
resolved "https://registry.yarnpkg.com/@tanstack/table-core/-/table-core-8.20.5.tgz#3974f0b090bed11243d4107283824167a395cf1d"
integrity sha512-P9dF7XbibHph2PFRz8gfBKEXEY/HJPOhym8CHmjF8y3q5mWpKx9xtZapXQUWCgkqvsK0R46Azuz+VaxD4Xl+Tg==
"@tanstack/table-core@8.21.3":
version "8.21.3"
resolved "https://registry.yarnpkg.com/@tanstack/table-core/-/table-core-8.21.3.tgz#2977727d8fc8dfa079112d9f4d4c019110f1732c"
integrity sha512-ldZXEhOBb8Is7xLs01fR3YEc3DERiz5silj8tnGkFZytt1abEvl/GhUmCE0PMLaMPTa3Jk4HbKmRlHmu+gCftg==
"@tanstack/virtual-core@3.11.2":
version "3.11.2"
resolved "https://registry.yarnpkg.com/@tanstack/virtual-core/-/virtual-core-3.11.2.tgz#00409e743ac4eea9afe5b7708594d5fcebb00212"
integrity sha512-vTtpNt7mKCiZ1pwU9hfKPhpdVO2sVzFQsxoVBGtOSHxlrRRzYr8iQ2TlwbAcRYCcEiZ9ECAM8kBzH0v2+VzfKw==
"@tanstack/virtual-core@3.13.12":
version "3.13.12"
resolved "https://registry.yarnpkg.com/@tanstack/virtual-core/-/virtual-core-3.13.12.tgz#1dff176df9cc8f93c78c5e46bcea11079b397578"
integrity sha512-1YBOJfRHV4sXUmWsFSf5rQor4Ss82G8dQWLRbnk3GA4jeP8hQt1hxXh0tmflpC0dz3VgEv/1+qwPyLeWkQuPFA==
"@tanstack/virtual-core@3.13.22":
version "3.13.22"
resolved "https://registry.yarnpkg.com/@tanstack/virtual-core/-/virtual-core-3.13.22.tgz#660a2cd048510125a4da898e5a659d53166f51af"
integrity sha512-isuUGKsc5TAPDoHSbWTbl1SCil54zOS2MiWz/9GCWHPUQOvNTQx8qJEWC7UWR0lShhbK0Lmkcf0SZYxvch7G3g==
"@testing-library/dom@^8.5.0":
version "8.20.0"
resolved "https://registry.npmjs.org/@testing-library/dom/-/dom-8.20.0.tgz"

View File

@@ -144,12 +144,12 @@ func (module *module) DeleteRole(ctx context.Context, orgID valuer.UUID, id valu
return err
}
err = module.authz.Revoke(ctx, orgID, []string{role.Name}, authtypes.MustNewSubject(authtypes.TypeableServiceAccount, id.String(), orgID, nil))
err = module.store.DeleteServiceAccountRole(ctx, serviceAccount.ID, roleID)
if err != nil {
return err
}
err = module.store.DeleteServiceAccountRole(ctx, serviceAccount.ID, roleID)
err = module.authz.Revoke(ctx, orgID, []string{role.Name}, authtypes.MustNewSubject(authtypes.TypeableServiceAccount, id.String(), orgID, nil))
if err != nil {
return err
}
@@ -386,12 +386,12 @@ func (module *module) setRole(ctx context.Context, orgID valuer.UUID, id valuer.
return err
}
err = module.authz.Grant(ctx, orgID, []string{role.Name}, authtypes.MustNewSubject(authtypes.TypeableServiceAccount, id.String(), orgID, nil))
err = module.store.CreateServiceAccountRole(ctx, serviceAccountRole)
if err != nil {
return err
}
err = module.store.CreateServiceAccountRole(ctx, serviceAccountRole)
err = module.authz.Grant(ctx, orgID, []string{role.Name}, authtypes.MustNewSubject(authtypes.TypeableServiceAccount, id.String(), orgID, nil))
if err != nil {
return err
}

View File

@@ -40,6 +40,7 @@ type querier struct {
promEngine prometheus.Prometheus
traceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation]
logStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation]
auditStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation]
metricStmtBuilder qbtypes.StatementBuilder[qbtypes.MetricAggregation]
meterStmtBuilder qbtypes.StatementBuilder[qbtypes.MetricAggregation]
traceOperatorStmtBuilder qbtypes.TraceOperatorStatementBuilder
@@ -56,6 +57,7 @@ func New(
promEngine prometheus.Prometheus,
traceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation],
logStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation],
auditStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation],
metricStmtBuilder qbtypes.StatementBuilder[qbtypes.MetricAggregation],
meterStmtBuilder qbtypes.StatementBuilder[qbtypes.MetricAggregation],
traceOperatorStmtBuilder qbtypes.TraceOperatorStatementBuilder,
@@ -69,6 +71,7 @@ func New(
promEngine: promEngine,
traceStmtBuilder: traceStmtBuilder,
logStmtBuilder: logStmtBuilder,
auditStmtBuilder: auditStmtBuilder,
metricStmtBuilder: metricStmtBuilder,
meterStmtBuilder: meterStmtBuilder,
traceOperatorStmtBuilder: traceOperatorStmtBuilder,
@@ -361,7 +364,11 @@ func (q *querier) QueryRange(ctx context.Context, orgID valuer.UUID, req *qbtype
case qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]:
spec.ShiftBy = extractShiftFromBuilderQuery(spec)
timeRange := adjustTimeRangeForShift(spec, qbtypes.TimeRange{From: req.Start, To: req.End}, req.RequestType)
bq := newBuilderQuery(q.logger, q.telemetryStore, q.logStmtBuilder, spec, timeRange, req.RequestType, tmplVars)
stmtBuilder := q.logStmtBuilder
if spec.Source == telemetrytypes.SourceAudit {
stmtBuilder = q.auditStmtBuilder
}
bq := newBuilderQuery(q.logger, q.telemetryStore, stmtBuilder, spec, timeRange, req.RequestType, tmplVars)
queries[spec.Name] = bq
steps[spec.Name] = spec.StepInterval
case qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]:
@@ -550,7 +557,11 @@ func (q *querier) QueryRawStream(ctx context.Context, orgID valuer.UUID, req *qb
case <-tick:
// timestamp end is not specified here
timeRange := adjustTimeRangeForShift(spec, qbtypes.TimeRange{From: tsStart}, req.RequestType)
bq := newBuilderQuery(q.logger, q.telemetryStore, q.logStmtBuilder, spec, timeRange, req.RequestType, map[string]qbtypes.VariableItem{
liveTailStmtBuilder := q.logStmtBuilder
if spec.Source == telemetrytypes.SourceAudit {
liveTailStmtBuilder = q.auditStmtBuilder
}
bq := newBuilderQuery(q.logger, q.telemetryStore, liveTailStmtBuilder, spec, timeRange, req.RequestType, map[string]qbtypes.VariableItem{
"id": {
Value: updatedLogID,
},
@@ -850,7 +861,11 @@ func (q *querier) createRangedQuery(originalQuery qbtypes.Query, timeRange qbtyp
specCopy := qt.spec.Copy()
specCopy.ShiftBy = extractShiftFromBuilderQuery(specCopy)
adjustedTimeRange := adjustTimeRangeForShift(specCopy, timeRange, qt.kind)
return newBuilderQuery(q.logger, q.telemetryStore, q.logStmtBuilder, specCopy, adjustedTimeRange, qt.kind, qt.variables)
shiftStmtBuilder := q.logStmtBuilder
if qt.spec.Source == telemetrytypes.SourceAudit {
shiftStmtBuilder = q.auditStmtBuilder
}
return newBuilderQuery(q.logger, q.telemetryStore, shiftStmtBuilder, specCopy, adjustedTimeRange, qt.kind, qt.variables)
case *builderQuery[qbtypes.MetricAggregation]:
specCopy := qt.spec.Copy()

View File

@@ -10,6 +10,7 @@ import (
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/querybuilder/resourcefilter"
"github.com/SigNoz/signoz/pkg/telemetryaudit"
"github.com/SigNoz/signoz/pkg/telemetrylogs"
"github.com/SigNoz/signoz/pkg/telemetrymetadata"
"github.com/SigNoz/signoz/pkg/telemetrymeter"
@@ -64,6 +65,11 @@ func newProvider(
telemetrylogs.TagAttributesV2TableName,
telemetrylogs.LogAttributeKeysTblName,
telemetrylogs.LogResourceKeysTblName,
telemetryaudit.DBName,
telemetryaudit.LogsTableName,
telemetryaudit.TagAttributesTableName,
telemetryaudit.LogAttributeKeysTblName,
telemetryaudit.LogResourceKeysTblName,
telemetrymetadata.DBName,
telemetrymetadata.AttributesMetadataLocalTableName,
telemetrymetadata.ColumnEvolutionMetadataTableName,
@@ -133,6 +139,35 @@ func newProvider(
telemetrylogs.GetBodyJSONKey,
)
// Create audit statement builder
auditFieldMapper := telemetryaudit.NewFieldMapper()
auditConditionBuilder := telemetryaudit.NewConditionBuilder(auditFieldMapper)
auditResourceFilterStmtBuilder := resourcefilter.NewLogResourceFilterStatementBuilder(
settings,
resourceFilterFieldMapper,
resourceFilterConditionBuilder,
telemetryMetadataStore,
telemetryaudit.DefaultFullTextColumn,
nil,
)
auditAggExprRewriter := querybuilder.NewAggExprRewriter(
settings,
telemetryaudit.DefaultFullTextColumn,
auditFieldMapper,
auditConditionBuilder,
nil,
)
auditStmtBuilder := telemetryaudit.NewAuditQueryStatementBuilder(
settings,
telemetryMetadataStore,
auditFieldMapper,
auditConditionBuilder,
auditResourceFilterStmtBuilder,
auditAggExprRewriter,
telemetryaudit.DefaultFullTextColumn,
nil,
)
// Create metric statement builder
metricFieldMapper := telemetrymetrics.NewFieldMapper()
metricConditionBuilder := telemetrymetrics.NewConditionBuilder(metricFieldMapper)
@@ -169,6 +204,7 @@ func newProvider(
prometheus,
traceStmtBuilder,
logStmtBuilder,
auditStmtBuilder,
metricStmtBuilder,
meterStmtBuilder,
traceOperatorStmtBuilder,

View File

@@ -47,6 +47,7 @@ func prepareQuerierForMetrics(t *testing.T, telemetryStore telemetrystore.Teleme
nil, // prometheus
nil, // traceStmtBuilder
nil, // logStmtBuilder
nil, // auditStmtBuilder
metricStmtBuilder,
nil, // meterStmtBuilder
nil, // traceOperatorStmtBuilder
@@ -104,6 +105,7 @@ func prepareQuerierForLogs(telemetryStore telemetrystore.TelemetryStore, keysMap
nil, // prometheus
nil, // traceStmtBuilder
logStmtBuilder, // logStmtBuilder
nil, // auditStmtBuilder
nil, // metricStmtBuilder
nil, // meterStmtBuilder
nil, // traceOperatorStmtBuilder
@@ -154,6 +156,7 @@ func prepareQuerierForTraces(telemetryStore telemetrystore.TelemetryStore, keysM
nil, // prometheus
traceStmtBuilder, // traceStmtBuilder
nil, // logStmtBuilder
nil, // auditStmtBuilder
nil, // metricStmtBuilder
nil, // meterStmtBuilder
nil, // traceOperatorStmtBuilder

View File

@@ -33,6 +33,7 @@ import (
"github.com/SigNoz/signoz/pkg/sqlschema"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/statsreporter"
"github.com/SigNoz/signoz/pkg/telemetryaudit"
"github.com/SigNoz/signoz/pkg/telemetrylogs"
"github.com/SigNoz/signoz/pkg/telemetrymetadata"
"github.com/SigNoz/signoz/pkg/telemetrymeter"
@@ -395,6 +396,11 @@ func New(
telemetrylogs.TagAttributesV2TableName,
telemetrylogs.LogAttributeKeysTblName,
telemetrylogs.LogResourceKeysTblName,
telemetryaudit.DBName,
telemetryaudit.LogsTableName,
telemetryaudit.TagAttributesTableName,
telemetryaudit.LogAttributeKeysTblName,
telemetryaudit.LogResourceKeysTblName,
telemetrymetadata.DBName,
telemetrymetadata.AttributesMetadataLocalTableName,
telemetrymetadata.ColumnEvolutionMetadataTableName,

View File

@@ -53,7 +53,7 @@ func newConfig() factory.Config {
},
Sqlite: SqliteConfig{
Path: "/var/lib/signoz/signoz.db",
Mode: "wal",
Mode: "delete",
BusyTimeout: 10000 * time.Millisecond, // increasing the defaults from https://github.com/mattn/go-sqlite3/blob/master/sqlite3.go#L1098 because of transpilation from C to GO
TransactionMode: "deferred",
},

View File

@@ -0,0 +1,204 @@
package telemetryaudit
import (
"context"
"fmt"
schema "github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/querybuilder"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/huandu/go-sqlbuilder"
)
type conditionBuilder struct {
fm qbtypes.FieldMapper
}
func NewConditionBuilder(fm qbtypes.FieldMapper) *conditionBuilder {
return &conditionBuilder{fm: fm}
}
func (c *conditionBuilder) conditionFor(
ctx context.Context,
startNs, endNs uint64,
key *telemetrytypes.TelemetryFieldKey,
operator qbtypes.FilterOperator,
value any,
sb *sqlbuilder.SelectBuilder,
) (string, error) {
columns, err := c.fm.ColumnFor(ctx, startNs, endNs, key)
if err != nil {
return "", err
}
if operator.IsStringSearchOperator() {
value = querybuilder.FormatValueForContains(value)
}
fieldExpression, err := c.fm.FieldFor(ctx, startNs, endNs, key)
if err != nil {
return "", err
}
fieldExpression, value = querybuilder.DataTypeCollisionHandledFieldName(key, value, fieldExpression, operator)
switch operator {
case qbtypes.FilterOperatorEqual:
return sb.E(fieldExpression, value), nil
case qbtypes.FilterOperatorNotEqual:
return sb.NE(fieldExpression, value), nil
case qbtypes.FilterOperatorGreaterThan:
return sb.G(fieldExpression, value), nil
case qbtypes.FilterOperatorGreaterThanOrEq:
return sb.GE(fieldExpression, value), nil
case qbtypes.FilterOperatorLessThan:
return sb.LT(fieldExpression, value), nil
case qbtypes.FilterOperatorLessThanOrEq:
return sb.LE(fieldExpression, value), nil
case qbtypes.FilterOperatorLike:
return sb.Like(fieldExpression, value), nil
case qbtypes.FilterOperatorNotLike:
return sb.NotLike(fieldExpression, value), nil
case qbtypes.FilterOperatorILike:
return sb.ILike(fieldExpression, value), nil
case qbtypes.FilterOperatorNotILike:
return sb.NotILike(fieldExpression, value), nil
case qbtypes.FilterOperatorContains:
return sb.ILike(fieldExpression, fmt.Sprintf("%%%s%%", value)), nil
case qbtypes.FilterOperatorNotContains:
return sb.NotILike(fieldExpression, fmt.Sprintf("%%%s%%", value)), nil
case qbtypes.FilterOperatorRegexp:
return fmt.Sprintf(`match(%s, %s)`, sqlbuilder.Escape(fieldExpression), sb.Var(value)), nil
case qbtypes.FilterOperatorNotRegexp:
return fmt.Sprintf(`NOT match(%s, %s)`, sqlbuilder.Escape(fieldExpression), sb.Var(value)), nil
case qbtypes.FilterOperatorBetween:
values, ok := value.([]any)
if !ok {
return "", qbtypes.ErrBetweenValues
}
if len(values) != 2 {
return "", qbtypes.ErrBetweenValues
}
return sb.Between(fieldExpression, values[0], values[1]), nil
case qbtypes.FilterOperatorNotBetween:
values, ok := value.([]any)
if !ok {
return "", qbtypes.ErrBetweenValues
}
if len(values) != 2 {
return "", qbtypes.ErrBetweenValues
}
return sb.NotBetween(fieldExpression, values[0], values[1]), nil
case qbtypes.FilterOperatorIn:
values, ok := value.([]any)
if !ok {
return "", qbtypes.ErrInValues
}
conditions := []string{}
for _, value := range values {
conditions = append(conditions, sb.E(fieldExpression, value))
}
return sb.Or(conditions...), nil
case qbtypes.FilterOperatorNotIn:
values, ok := value.([]any)
if !ok {
return "", qbtypes.ErrInValues
}
conditions := []string{}
for _, value := range values {
conditions = append(conditions, sb.NE(fieldExpression, value))
}
return sb.And(conditions...), nil
case qbtypes.FilterOperatorExists, qbtypes.FilterOperatorNotExists:
var value any
column := columns[0]
switch column.Type.GetType() {
case schema.ColumnTypeEnumJSON:
if operator == qbtypes.FilterOperatorExists {
return sb.IsNotNull(fieldExpression), nil
}
return sb.IsNull(fieldExpression), nil
case schema.ColumnTypeEnumLowCardinality:
switch elementType := column.Type.(schema.LowCardinalityColumnType).ElementType; elementType.GetType() {
case schema.ColumnTypeEnumString:
value = ""
if operator == qbtypes.FilterOperatorExists {
return sb.NE(fieldExpression, value), nil
}
return sb.E(fieldExpression, value), nil
default:
return "", errors.NewInvalidInputf(errors.CodeInvalidInput, "exists operator is not supported for low cardinality column type %s", elementType)
}
case schema.ColumnTypeEnumString:
value = ""
if operator == qbtypes.FilterOperatorExists {
return sb.NE(fieldExpression, value), nil
}
return sb.E(fieldExpression, value), nil
case schema.ColumnTypeEnumUInt64, schema.ColumnTypeEnumUInt32, schema.ColumnTypeEnumUInt8:
value = 0
if operator == qbtypes.FilterOperatorExists {
return sb.NE(fieldExpression, value), nil
}
return sb.E(fieldExpression, value), nil
case schema.ColumnTypeEnumMap:
keyType := column.Type.(schema.MapColumnType).KeyType
if _, ok := keyType.(schema.LowCardinalityColumnType); !ok {
return "", errors.NewInvalidInputf(errors.CodeInvalidInput, "key type %s is not supported for map column type %s", keyType, column.Type)
}
switch valueType := column.Type.(schema.MapColumnType).ValueType; valueType.GetType() {
case schema.ColumnTypeEnumString, schema.ColumnTypeEnumBool, schema.ColumnTypeEnumFloat64:
leftOperand := fmt.Sprintf("mapContains(%s, '%s')", column.Name, key.Name)
if key.Materialized {
leftOperand = telemetrytypes.FieldKeyToMaterializedColumnNameForExists(key)
}
if operator == qbtypes.FilterOperatorExists {
return sb.E(leftOperand, true), nil
}
return sb.NE(leftOperand, true), nil
default:
return "", errors.NewInvalidInputf(errors.CodeInvalidInput, "exists operator is not supported for map column type %s", valueType)
}
default:
return "", errors.NewInvalidInputf(errors.CodeInvalidInput, "exists operator is not supported for column type %s", column.Type)
}
}
return "", errors.NewInvalidInputf(errors.CodeInvalidInput, "unsupported operator: %v", operator)
}
func (c *conditionBuilder) ConditionFor(
ctx context.Context,
startNs uint64,
endNs uint64,
key *telemetrytypes.TelemetryFieldKey,
operator qbtypes.FilterOperator,
value any,
sb *sqlbuilder.SelectBuilder,
) (string, error) {
condition, err := c.conditionFor(ctx, startNs, endNs, key, operator, value, sb)
if err != nil {
return "", err
}
buildExistCondition := operator.AddDefaultExistsFilter()
switch key.FieldContext {
case telemetrytypes.FieldContextLog, telemetrytypes.FieldContextScope:
return condition, nil
case telemetrytypes.FieldContextResource, telemetrytypes.FieldContextAttribute:
// build exist condition for resource and attribute fields based on filter operator
}
if buildExistCondition {
existsCondition, err := c.conditionFor(ctx, startNs, endNs, key, qbtypes.FilterOperatorExists, nil, sb)
if err != nil {
return "", err
}
return sb.And(condition, existsCondition), nil
}
return condition, nil
}

128
pkg/telemetryaudit/const.go Normal file
View File

@@ -0,0 +1,128 @@
package telemetryaudit
import (
schema "github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
)
const (
// Internal Columns.
IDColumn = "id"
TimestampBucketStartColumn = "ts_bucket_start"
ResourceFingerPrintColumn = "resource_fingerprint"
// Intrinsic Columns.
TimestampColumn = "timestamp"
ObservedTimestampColumn = "observed_timestamp"
BodyColumn = "body"
EventNameColumn = "event_name"
TraceIDColumn = "trace_id"
SpanIDColumn = "span_id"
TraceFlagsColumn = "trace_flags"
SeverityTextColumn = "severity_text"
SeverityNumberColumn = "severity_number"
ScopeNameColumn = "scope_name"
ScopeVersionColumn = "scope_version"
// Contextual Columns.
AttributesStringColumn = "attributes_string"
AttributesNumberColumn = "attributes_number"
AttributesBoolColumn = "attributes_bool"
ScopeStringColumn = "scope_string"
)
var (
DefaultFullTextColumn = &telemetrytypes.TelemetryFieldKey{
Name: "body",
Signal: telemetrytypes.SignalLogs,
FieldContext: telemetrytypes.FieldContextLog,
FieldDataType: telemetrytypes.FieldDataTypeString,
}
IntrinsicFields = map[string]telemetrytypes.TelemetryFieldKey{
"body": {
Name: "body",
Signal: telemetrytypes.SignalLogs,
FieldContext: telemetrytypes.FieldContextLog,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
"trace_id": {
Name: "trace_id",
Signal: telemetrytypes.SignalLogs,
FieldContext: telemetrytypes.FieldContextLog,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
"span_id": {
Name: "span_id",
Signal: telemetrytypes.SignalLogs,
FieldContext: telemetrytypes.FieldContextLog,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
"trace_flags": {
Name: "trace_flags",
Signal: telemetrytypes.SignalLogs,
FieldContext: telemetrytypes.FieldContextLog,
FieldDataType: telemetrytypes.FieldDataTypeNumber,
},
"severity_text": {
Name: "severity_text",
Signal: telemetrytypes.SignalLogs,
FieldContext: telemetrytypes.FieldContextLog,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
"severity_number": {
Name: "severity_number",
Signal: telemetrytypes.SignalLogs,
FieldContext: telemetrytypes.FieldContextLog,
FieldDataType: telemetrytypes.FieldDataTypeNumber,
},
"event_name": {
Name: "event_name",
Signal: telemetrytypes.SignalLogs,
FieldContext: telemetrytypes.FieldContextLog,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
}
DefaultSortingOrder = []qbtypes.OrderBy{
{
Key: qbtypes.OrderByKey{
TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{
Name: TimestampColumn,
},
},
Direction: qbtypes.OrderDirectionDesc,
},
{
Key: qbtypes.OrderByKey{
TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{
Name: IDColumn,
},
},
Direction: qbtypes.OrderDirectionDesc,
},
}
)
var logsV2Columns = map[string]*schema.Column{
"ts_bucket_start": {Name: "ts_bucket_start", Type: schema.ColumnTypeUInt64},
"resource_fingerprint": {Name: "resource_fingerprint", Type: schema.ColumnTypeString},
"timestamp": {Name: "timestamp", Type: schema.ColumnTypeUInt64},
"observed_timestamp": {Name: "observed_timestamp", Type: schema.ColumnTypeUInt64},
"id": {Name: "id", Type: schema.ColumnTypeString},
"trace_id": {Name: "trace_id", Type: schema.ColumnTypeString},
"span_id": {Name: "span_id", Type: schema.ColumnTypeString},
"trace_flags": {Name: "trace_flags", Type: schema.ColumnTypeUInt32},
"severity_text": {Name: "severity_text", Type: schema.LowCardinalityColumnType{ElementType: schema.ColumnTypeString}},
"severity_number": {Name: "severity_number", Type: schema.ColumnTypeUInt8},
"body": {Name: "body", Type: schema.ColumnTypeString},
"attributes_string": {Name: "attributes_string", Type: schema.MapColumnType{KeyType: schema.LowCardinalityColumnType{ElementType: schema.ColumnTypeString}, ValueType: schema.ColumnTypeString}},
"attributes_number": {Name: "attributes_number", Type: schema.MapColumnType{KeyType: schema.LowCardinalityColumnType{ElementType: schema.ColumnTypeString}, ValueType: schema.ColumnTypeFloat64}},
"attributes_bool": {Name: "attributes_bool", Type: schema.MapColumnType{KeyType: schema.LowCardinalityColumnType{ElementType: schema.ColumnTypeString}, ValueType: schema.ColumnTypeBool}},
"resource": {Name: "resource", Type: schema.JSONColumnType{}},
"event_name": {Name: "event_name", Type: schema.ColumnTypeString},
"scope_name": {Name: "scope_name", Type: schema.ColumnTypeString},
"scope_version": {Name: "scope_version", Type: schema.ColumnTypeString},
"scope_string": {Name: "scope_string", Type: schema.MapColumnType{KeyType: schema.LowCardinalityColumnType{ElementType: schema.ColumnTypeString}, ValueType: schema.ColumnTypeString}},
}

View File

@@ -0,0 +1,155 @@
package telemetryaudit
import (
"context"
"fmt"
"strings"
schema "github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator"
"github.com/SigNoz/signoz/pkg/errors"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/huandu/go-sqlbuilder"
"golang.org/x/exp/maps"
)
type fieldMapper struct{}
func NewFieldMapper() qbtypes.FieldMapper {
return &fieldMapper{}
}
func (m *fieldMapper) getColumn(_ context.Context, key *telemetrytypes.TelemetryFieldKey) ([]*schema.Column, error) {
switch key.FieldContext {
case telemetrytypes.FieldContextResource:
return []*schema.Column{logsV2Columns["resource"]}, nil
case telemetrytypes.FieldContextScope:
switch key.Name {
case "name", "scope.name", "scope_name":
return []*schema.Column{logsV2Columns["scope_name"]}, nil
case "version", "scope.version", "scope_version":
return []*schema.Column{logsV2Columns["scope_version"]}, nil
}
return []*schema.Column{logsV2Columns["scope_string"]}, nil
case telemetrytypes.FieldContextAttribute:
switch key.FieldDataType {
case telemetrytypes.FieldDataTypeString:
return []*schema.Column{logsV2Columns["attributes_string"]}, nil
case telemetrytypes.FieldDataTypeInt64, telemetrytypes.FieldDataTypeFloat64, telemetrytypes.FieldDataTypeNumber:
return []*schema.Column{logsV2Columns["attributes_number"]}, nil
case telemetrytypes.FieldDataTypeBool:
return []*schema.Column{logsV2Columns["attributes_bool"]}, nil
}
case telemetrytypes.FieldContextLog, telemetrytypes.FieldContextUnspecified:
col, ok := logsV2Columns[key.Name]
if !ok {
return nil, qbtypes.ErrColumnNotFound
}
return []*schema.Column{col}, nil
}
return nil, qbtypes.ErrColumnNotFound
}
func (m *fieldMapper) FieldFor(ctx context.Context, _, _ uint64, key *telemetrytypes.TelemetryFieldKey) (string, error) {
columns, err := m.getColumn(ctx, key)
if err != nil {
return "", err
}
exprs := []string{}
existExpr := []string{}
for _, column := range columns {
switch column.Type.GetType() {
case schema.ColumnTypeEnumJSON:
if key.FieldContext == telemetrytypes.FieldContextResource {
exprs = append(exprs, fmt.Sprintf("%s.`%s`::String", column.Name, key.Name))
existExpr = append(existExpr, fmt.Sprintf("%s.`%s` IS NOT NULL", column.Name, key.Name))
} else {
return "", errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "only resource context fields are supported for json columns in audit, got %s", key.FieldContext.String)
}
case schema.ColumnTypeEnumLowCardinality:
switch elementType := column.Type.(schema.LowCardinalityColumnType).ElementType; elementType.GetType() {
case schema.ColumnTypeEnumString:
exprs = append(exprs, column.Name)
default:
return "", errors.NewInvalidInputf(errors.CodeInvalidInput, "unsupported low cardinality element type %s", elementType)
}
case schema.ColumnTypeEnumString, schema.ColumnTypeEnumUInt64, schema.ColumnTypeEnumUInt32, schema.ColumnTypeEnumUInt8:
exprs = append(exprs, column.Name)
case schema.ColumnTypeEnumMap:
keyType := column.Type.(schema.MapColumnType).KeyType
if _, ok := keyType.(schema.LowCardinalityColumnType); !ok {
return "", errors.NewInvalidInputf(errors.CodeInvalidInput, "key type %s is not supported for map column type %s", keyType, column.Type)
}
switch valueType := column.Type.(schema.MapColumnType).ValueType; valueType.GetType() {
case schema.ColumnTypeEnumString, schema.ColumnTypeEnumBool, schema.ColumnTypeEnumFloat64:
if key.Materialized {
exprs = append(exprs, telemetrytypes.FieldKeyToMaterializedColumnName(key))
existExpr = append(existExpr, fmt.Sprintf("%s==true", telemetrytypes.FieldKeyToMaterializedColumnNameForExists(key)))
} else {
exprs = append(exprs, fmt.Sprintf("%s['%s']", column.Name, key.Name))
existExpr = append(existExpr, fmt.Sprintf("mapContains(%s, '%s')", column.Name, key.Name))
}
default:
return "", errors.NewInvalidInputf(errors.CodeInvalidInput, "unsupported map value type %s", valueType)
}
}
}
if len(exprs) == 1 {
return exprs[0], nil
} else if len(exprs) > 1 {
if len(existExpr) != len(exprs) {
return "", errors.New(errors.TypeInternal, errors.CodeInternal, "length of exist exprs doesn't match to that of exprs")
}
finalExprs := []string{}
for i, expr := range exprs {
finalExprs = append(finalExprs, fmt.Sprintf("%s, %s", existExpr[i], expr))
}
return "multiIf(" + strings.Join(finalExprs, ", ") + ", NULL)", nil
}
return columns[0].Name, nil
}
func (m *fieldMapper) ColumnFor(ctx context.Context, _, _ uint64, key *telemetrytypes.TelemetryFieldKey) ([]*schema.Column, error) {
return m.getColumn(ctx, key)
}
func (m *fieldMapper) ColumnExpressionFor(
ctx context.Context,
tsStart, tsEnd uint64,
field *telemetrytypes.TelemetryFieldKey,
keys map[string][]*telemetrytypes.TelemetryFieldKey,
) (string, error) {
fieldExpression, err := m.FieldFor(ctx, tsStart, tsEnd, field)
if errors.Is(err, qbtypes.ErrColumnNotFound) {
keysForField := keys[field.Name]
if len(keysForField) == 0 {
if _, ok := logsV2Columns[field.Name]; ok {
field.FieldContext = telemetrytypes.FieldContextLog
fieldExpression, _ = m.FieldFor(ctx, tsStart, tsEnd, field)
} else {
correction, found := telemetrytypes.SuggestCorrection(field.Name, maps.Keys(keys))
if found {
return "", errors.Wrap(err, errors.TypeInvalidInput, errors.CodeInvalidInput, correction)
}
return "", errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "field `%s` not found", field.Name)
}
} else if len(keysForField) == 1 {
fieldExpression, _ = m.FieldFor(ctx, tsStart, tsEnd, keysForField[0])
} else {
args := []string{}
for _, key := range keysForField {
fieldExpression, _ = m.FieldFor(ctx, tsStart, tsEnd, key)
args = append(args, fmt.Sprintf("toString(%s) != '', toString(%s)", fieldExpression, fieldExpression))
}
fieldExpression = fmt.Sprintf("multiIf(%s, NULL)", strings.Join(args, ", "))
}
}
return fmt.Sprintf("%s AS `%s`", sqlbuilder.Escape(fieldExpression), field.Name), nil
}

View File

@@ -0,0 +1,600 @@
package telemetryaudit
import (
"context"
"fmt"
"log/slog"
"strings"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/querybuilder"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/huandu/go-sqlbuilder"
)
type auditQueryStatementBuilder struct {
logger *slog.Logger
metadataStore telemetrytypes.MetadataStore
fm qbtypes.FieldMapper
cb qbtypes.ConditionBuilder
resourceFilterStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation]
aggExprRewriter qbtypes.AggExprRewriter
fullTextColumn *telemetrytypes.TelemetryFieldKey
jsonKeyToKey qbtypes.JsonKeyToFieldFunc
}
var _ qbtypes.StatementBuilder[qbtypes.LogAggregation] = (*auditQueryStatementBuilder)(nil)
func NewAuditQueryStatementBuilder(
settings factory.ProviderSettings,
metadataStore telemetrytypes.MetadataStore,
fieldMapper qbtypes.FieldMapper,
conditionBuilder qbtypes.ConditionBuilder,
resourceFilterStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation],
aggExprRewriter qbtypes.AggExprRewriter,
fullTextColumn *telemetrytypes.TelemetryFieldKey,
jsonKeyToKey qbtypes.JsonKeyToFieldFunc,
) *auditQueryStatementBuilder {
auditSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetryaudit")
return &auditQueryStatementBuilder{
logger: auditSettings.Logger(),
metadataStore: metadataStore,
fm: fieldMapper,
cb: conditionBuilder,
resourceFilterStmtBuilder: resourceFilterStmtBuilder,
aggExprRewriter: aggExprRewriter,
fullTextColumn: fullTextColumn,
jsonKeyToKey: jsonKeyToKey,
}
}
func (b *auditQueryStatementBuilder) Build(
ctx context.Context,
start uint64,
end uint64,
requestType qbtypes.RequestType,
query qbtypes.QueryBuilderQuery[qbtypes.LogAggregation],
variables map[string]qbtypes.VariableItem,
) (*qbtypes.Statement, error) {
start = querybuilder.ToNanoSecs(start)
end = querybuilder.ToNanoSecs(end)
keySelectors := getKeySelectors(query)
keys, _, err := b.metadataStore.GetKeysMulti(ctx, keySelectors)
if err != nil {
return nil, err
}
query = b.adjustKeys(ctx, keys, query, requestType)
q := sqlbuilder.NewSelectBuilder()
var stmt *qbtypes.Statement
switch requestType {
case qbtypes.RequestTypeRaw, qbtypes.RequestTypeRawStream:
stmt, err = b.buildListQuery(ctx, q, query, start, end, keys, variables)
case qbtypes.RequestTypeTimeSeries:
stmt, err = b.buildTimeSeriesQuery(ctx, q, query, start, end, keys, variables)
case qbtypes.RequestTypeScalar:
stmt, err = b.buildScalarQuery(ctx, q, query, start, end, keys, false, variables)
default:
return nil, errors.NewInvalidInputf(errors.CodeInvalidInput, "unsupported request type: %s", requestType)
}
if err != nil {
return nil, err
}
return stmt, nil
}
func getKeySelectors(query qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]) []*telemetrytypes.FieldKeySelector {
var keySelectors []*telemetrytypes.FieldKeySelector
for idx := range query.Aggregations {
aggExpr := query.Aggregations[idx]
selectors := querybuilder.QueryStringToKeysSelectors(aggExpr.Expression)
keySelectors = append(keySelectors, selectors...)
}
if query.Filter != nil && query.Filter.Expression != "" {
whereClauseSelectors := querybuilder.QueryStringToKeysSelectors(query.Filter.Expression)
keySelectors = append(keySelectors, whereClauseSelectors...)
}
for idx := range query.GroupBy {
groupBy := query.GroupBy[idx]
keySelectors = append(keySelectors, &telemetrytypes.FieldKeySelector{
Name: groupBy.Name,
Signal: telemetrytypes.SignalLogs,
FieldContext: groupBy.FieldContext,
FieldDataType: groupBy.FieldDataType,
})
}
for idx := range query.SelectFields {
selectField := query.SelectFields[idx]
keySelectors = append(keySelectors, &telemetrytypes.FieldKeySelector{
Name: selectField.Name,
Signal: telemetrytypes.SignalLogs,
FieldContext: selectField.FieldContext,
FieldDataType: selectField.FieldDataType,
})
}
for idx := range query.Order {
keySelectors = append(keySelectors, &telemetrytypes.FieldKeySelector{
Name: query.Order[idx].Key.Name,
Signal: telemetrytypes.SignalLogs,
FieldContext: query.Order[idx].Key.FieldContext,
FieldDataType: query.Order[idx].Key.FieldDataType,
})
}
for idx := range keySelectors {
keySelectors[idx].Signal = telemetrytypes.SignalLogs
keySelectors[idx].Source = telemetrytypes.SourceAudit
keySelectors[idx].SelectorMatchType = telemetrytypes.FieldSelectorMatchTypeExact
}
return keySelectors
}
func (b *auditQueryStatementBuilder) adjustKeys(ctx context.Context, keys map[string][]*telemetrytypes.TelemetryFieldKey, query qbtypes.QueryBuilderQuery[qbtypes.LogAggregation], requestType qbtypes.RequestType) qbtypes.QueryBuilderQuery[qbtypes.LogAggregation] {
keys["id"] = append([]*telemetrytypes.TelemetryFieldKey{{
Name: "id",
Signal: telemetrytypes.SignalLogs,
FieldContext: telemetrytypes.FieldContextLog,
FieldDataType: telemetrytypes.FieldDataTypeString,
}}, keys["id"]...)
keys["timestamp"] = append([]*telemetrytypes.TelemetryFieldKey{{
Name: "timestamp",
Signal: telemetrytypes.SignalLogs,
FieldContext: telemetrytypes.FieldContextLog,
FieldDataType: telemetrytypes.FieldDataTypeNumber,
}}, keys["timestamp"]...)
actions := querybuilder.AdjustKeysForAliasExpressions(&query, requestType)
actions = append(actions, querybuilder.AdjustDuplicateKeys(&query)...)
for idx := range query.SelectFields {
actions = append(actions, b.adjustKey(&query.SelectFields[idx], keys)...)
}
for idx := range query.GroupBy {
actions = append(actions, b.adjustKey(&query.GroupBy[idx].TelemetryFieldKey, keys)...)
}
for idx := range query.Order {
actions = append(actions, b.adjustKey(&query.Order[idx].Key.TelemetryFieldKey, keys)...)
}
for _, action := range actions {
b.logger.InfoContext(ctx, "key adjustment action", slog.String("action", action))
}
return query
}
func (b *auditQueryStatementBuilder) adjustKey(key *telemetrytypes.TelemetryFieldKey, keys map[string][]*telemetrytypes.TelemetryFieldKey) []string {
if _, ok := IntrinsicFields[key.Name]; ok {
intrinsicField := IntrinsicFields[key.Name]
return querybuilder.AdjustKey(key, keys, &intrinsicField)
}
return querybuilder.AdjustKey(key, keys, nil)
}
func (b *auditQueryStatementBuilder) buildListQuery(
ctx context.Context,
sb *sqlbuilder.SelectBuilder,
query qbtypes.QueryBuilderQuery[qbtypes.LogAggregation],
start, end uint64,
keys map[string][]*telemetrytypes.TelemetryFieldKey,
variables map[string]qbtypes.VariableItem,
) (*qbtypes.Statement, error) {
var (
cteFragments []string
cteArgs [][]any
)
if frag, args, err := b.maybeAttachResourceFilter(ctx, sb, query, start, end, variables); err != nil {
return nil, err
} else if frag != "" {
cteFragments = append(cteFragments, frag)
cteArgs = append(cteArgs, args)
}
sb.Select(TimestampColumn)
sb.SelectMore(IDColumn)
if len(query.SelectFields) == 0 {
sb.SelectMore(TraceIDColumn)
sb.SelectMore(SpanIDColumn)
sb.SelectMore(TraceFlagsColumn)
sb.SelectMore(SeverityTextColumn)
sb.SelectMore(SeverityNumberColumn)
sb.SelectMore(ScopeNameColumn)
sb.SelectMore(ScopeVersionColumn)
sb.SelectMore(BodyColumn)
sb.SelectMore(EventNameColumn)
sb.SelectMore(AttributesStringColumn)
sb.SelectMore(AttributesNumberColumn)
sb.SelectMore(AttributesBoolColumn)
sb.SelectMore(ScopeStringColumn)
} else {
for index := range query.SelectFields {
if query.SelectFields[index].Name == TimestampColumn || query.SelectFields[index].Name == IDColumn {
continue
}
colExpr, err := b.fm.ColumnExpressionFor(ctx, start, end, &query.SelectFields[index], keys)
if err != nil {
return nil, err
}
sb.SelectMore(colExpr)
}
}
sb.From(fmt.Sprintf("%s.%s", DBName, LogsTableName))
preparedWhereClause, err := b.addFilterCondition(ctx, sb, start, end, query, keys, variables)
if err != nil {
return nil, err
}
for _, orderBy := range query.Order {
colExpr, err := b.fm.ColumnExpressionFor(ctx, start, end, &orderBy.Key.TelemetryFieldKey, keys)
if err != nil {
return nil, err
}
sb.OrderBy(fmt.Sprintf("%s %s", colExpr, orderBy.Direction.StringValue()))
}
if query.Limit > 0 {
sb.Limit(query.Limit)
} else {
sb.Limit(100)
}
if query.Offset > 0 {
sb.Offset(query.Offset)
}
mainSQL, mainArgs := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
finalSQL := querybuilder.CombineCTEs(cteFragments) + mainSQL
finalArgs := querybuilder.PrependArgs(cteArgs, mainArgs)
stmt := &qbtypes.Statement{
Query: finalSQL,
Args: finalArgs,
}
if preparedWhereClause != nil {
stmt.Warnings = preparedWhereClause.Warnings
stmt.WarningsDocURL = preparedWhereClause.WarningsDocURL
}
return stmt, nil
}
func (b *auditQueryStatementBuilder) buildTimeSeriesQuery(
ctx context.Context,
sb *sqlbuilder.SelectBuilder,
query qbtypes.QueryBuilderQuery[qbtypes.LogAggregation],
start, end uint64,
keys map[string][]*telemetrytypes.TelemetryFieldKey,
variables map[string]qbtypes.VariableItem,
) (*qbtypes.Statement, error) {
var (
cteFragments []string
cteArgs [][]any
)
if frag, args, err := b.maybeAttachResourceFilter(ctx, sb, query, start, end, variables); err != nil {
return nil, err
} else if frag != "" {
cteFragments = append(cteFragments, frag)
cteArgs = append(cteArgs, args)
}
sb.SelectMore(fmt.Sprintf(
"toStartOfInterval(fromUnixTimestamp64Nano(timestamp), INTERVAL %d SECOND) AS ts",
int64(query.StepInterval.Seconds()),
))
var allGroupByArgs []any
fieldNames := make([]string, 0, len(query.GroupBy))
for _, gb := range query.GroupBy {
expr, args, err := querybuilder.CollisionHandledFinalExpr(ctx, start, end, &gb.TelemetryFieldKey, b.fm, b.cb, keys, telemetrytypes.FieldDataTypeString, b.jsonKeyToKey)
if err != nil {
return nil, err
}
colExpr := fmt.Sprintf("toString(%s) AS `%s`", expr, gb.Name)
allGroupByArgs = append(allGroupByArgs, args...)
sb.SelectMore(colExpr)
fieldNames = append(fieldNames, fmt.Sprintf("`%s`", gb.Name))
}
allAggChArgs := make([]any, 0)
for i, agg := range query.Aggregations {
rewritten, chArgs, err := b.aggExprRewriter.Rewrite(ctx, start, end, agg.Expression, uint64(query.StepInterval.Seconds()), keys)
if err != nil {
return nil, err
}
allAggChArgs = append(allAggChArgs, chArgs...)
sb.SelectMore(fmt.Sprintf("%s AS __result_%d", rewritten, i))
}
sb.From(fmt.Sprintf("%s.%s", DBName, LogsTableName))
preparedWhereClause, err := b.addFilterCondition(ctx, sb, start, end, query, keys, variables)
if err != nil {
return nil, err
}
var finalSQL string
var finalArgs []any
if query.Limit > 0 && len(query.GroupBy) > 0 {
cteSB := sqlbuilder.NewSelectBuilder()
cteStmt, err := b.buildScalarQuery(ctx, cteSB, query, start, end, keys, true, variables)
if err != nil {
return nil, err
}
cteFragments = append(cteFragments, fmt.Sprintf("__limit_cte AS (%s)", cteStmt.Query))
cteArgs = append(cteArgs, cteStmt.Args)
tuple := fmt.Sprintf("(%s)", strings.Join(fieldNames, ", "))
sb.Where(fmt.Sprintf("%s GLOBAL IN (SELECT %s FROM __limit_cte)", tuple, strings.Join(fieldNames, ", ")))
sb.GroupBy("ts")
sb.GroupBy(querybuilder.GroupByKeys(query.GroupBy)...)
if query.Having != nil && query.Having.Expression != "" {
rewriter := querybuilder.NewHavingExpressionRewriter()
rewrittenExpr, err := rewriter.RewriteForLogs(query.Having.Expression, query.Aggregations)
if err != nil {
return nil, err
}
sb.Having(rewrittenExpr)
}
if len(query.Order) != 0 {
for _, orderBy := range query.Order {
_, ok := aggOrderBy(orderBy, query)
if !ok {
sb.OrderBy(fmt.Sprintf("`%s` %s", orderBy.Key.Name, orderBy.Direction.StringValue()))
}
}
sb.OrderBy("ts desc")
}
combinedArgs := append(allGroupByArgs, allAggChArgs...)
mainSQL, mainArgs := sb.BuildWithFlavor(sqlbuilder.ClickHouse, combinedArgs...)
finalSQL = querybuilder.CombineCTEs(cteFragments) + mainSQL
finalArgs = querybuilder.PrependArgs(cteArgs, mainArgs)
} else {
sb.GroupBy("ts")
sb.GroupBy(querybuilder.GroupByKeys(query.GroupBy)...)
if query.Having != nil && query.Having.Expression != "" {
rewriter := querybuilder.NewHavingExpressionRewriter()
rewrittenExpr, err := rewriter.RewriteForLogs(query.Having.Expression, query.Aggregations)
if err != nil {
return nil, err
}
sb.Having(rewrittenExpr)
}
if len(query.Order) != 0 {
for _, orderBy := range query.Order {
_, ok := aggOrderBy(orderBy, query)
if !ok {
sb.OrderBy(fmt.Sprintf("`%s` %s", orderBy.Key.Name, orderBy.Direction.StringValue()))
}
}
sb.OrderBy("ts desc")
}
combinedArgs := append(allGroupByArgs, allAggChArgs...)
mainSQL, mainArgs := sb.BuildWithFlavor(sqlbuilder.ClickHouse, combinedArgs...)
finalSQL = querybuilder.CombineCTEs(cteFragments) + mainSQL
finalArgs = querybuilder.PrependArgs(cteArgs, mainArgs)
}
stmt := &qbtypes.Statement{
Query: finalSQL,
Args: finalArgs,
}
if preparedWhereClause != nil {
stmt.Warnings = preparedWhereClause.Warnings
stmt.WarningsDocURL = preparedWhereClause.WarningsDocURL
}
return stmt, nil
}
func (b *auditQueryStatementBuilder) buildScalarQuery(
ctx context.Context,
sb *sqlbuilder.SelectBuilder,
query qbtypes.QueryBuilderQuery[qbtypes.LogAggregation],
start, end uint64,
keys map[string][]*telemetrytypes.TelemetryFieldKey,
skipResourceCTE bool,
variables map[string]qbtypes.VariableItem,
) (*qbtypes.Statement, error) {
var (
cteFragments []string
cteArgs [][]any
)
if frag, args, err := b.maybeAttachResourceFilter(ctx, sb, query, start, end, variables); err != nil {
return nil, err
} else if frag != "" && !skipResourceCTE {
cteFragments = append(cteFragments, frag)
cteArgs = append(cteArgs, args)
}
allAggChArgs := []any{}
var allGroupByArgs []any
for _, gb := range query.GroupBy {
expr, args, err := querybuilder.CollisionHandledFinalExpr(ctx, start, end, &gb.TelemetryFieldKey, b.fm, b.cb, keys, telemetrytypes.FieldDataTypeString, b.jsonKeyToKey)
if err != nil {
return nil, err
}
colExpr := fmt.Sprintf("toString(%s) AS `%s`", expr, gb.Name)
allGroupByArgs = append(allGroupByArgs, args...)
sb.SelectMore(colExpr)
}
rateInterval := (end - start) / querybuilder.NsToSeconds
if len(query.Aggregations) > 0 {
for idx := range query.Aggregations {
aggExpr := query.Aggregations[idx]
rewritten, chArgs, err := b.aggExprRewriter.Rewrite(ctx, start, end, aggExpr.Expression, rateInterval, keys)
if err != nil {
return nil, err
}
allAggChArgs = append(allAggChArgs, chArgs...)
sb.SelectMore(fmt.Sprintf("%s AS __result_%d", rewritten, idx))
}
}
sb.From(fmt.Sprintf("%s.%s", DBName, LogsTableName))
preparedWhereClause, err := b.addFilterCondition(ctx, sb, start, end, query, keys, variables)
if err != nil {
return nil, err
}
sb.GroupBy(querybuilder.GroupByKeys(query.GroupBy)...)
if query.Having != nil && query.Having.Expression != "" {
rewriter := querybuilder.NewHavingExpressionRewriter()
rewrittenExpr, err := rewriter.RewriteForLogs(query.Having.Expression, query.Aggregations)
if err != nil {
return nil, err
}
sb.Having(rewrittenExpr)
}
for _, orderBy := range query.Order {
idx, ok := aggOrderBy(orderBy, query)
if ok {
sb.OrderBy(fmt.Sprintf("__result_%d %s", idx, orderBy.Direction.StringValue()))
} else {
sb.OrderBy(fmt.Sprintf("`%s` %s", orderBy.Key.Name, orderBy.Direction.StringValue()))
}
}
if len(query.Order) == 0 {
sb.OrderBy("__result_0 DESC")
}
if query.Limit > 0 {
sb.Limit(query.Limit)
}
combinedArgs := append(allGroupByArgs, allAggChArgs...)
mainSQL, mainArgs := sb.BuildWithFlavor(sqlbuilder.ClickHouse, combinedArgs...)
finalSQL := querybuilder.CombineCTEs(cteFragments) + mainSQL
finalArgs := querybuilder.PrependArgs(cteArgs, mainArgs)
stmt := &qbtypes.Statement{
Query: finalSQL,
Args: finalArgs,
}
if preparedWhereClause != nil {
stmt.Warnings = preparedWhereClause.Warnings
stmt.WarningsDocURL = preparedWhereClause.WarningsDocURL
}
return stmt, nil
}
func (b *auditQueryStatementBuilder) addFilterCondition(
ctx context.Context,
sb *sqlbuilder.SelectBuilder,
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.LogAggregation],
keys map[string][]*telemetrytypes.TelemetryFieldKey,
variables map[string]qbtypes.VariableItem,
) (*querybuilder.PreparedWhereClause, error) {
var preparedWhereClause *querybuilder.PreparedWhereClause
var err error
if query.Filter != nil && query.Filter.Expression != "" {
preparedWhereClause, err = querybuilder.PrepareWhereClause(query.Filter.Expression, querybuilder.FilterExprVisitorOpts{
Context: ctx,
Logger: b.logger,
FieldMapper: b.fm,
ConditionBuilder: b.cb,
FieldKeys: keys,
SkipResourceFilter: true,
FullTextColumn: b.fullTextColumn,
JsonKeyToKey: b.jsonKeyToKey,
Variables: variables,
StartNs: start,
EndNs: end,
})
if err != nil {
return nil, err
}
}
if preparedWhereClause != nil {
sb.AddWhereClause(preparedWhereClause.WhereClause)
}
startBucket := start/querybuilder.NsToSeconds - querybuilder.BucketAdjustment
var endBucket uint64
if end != 0 {
endBucket = end / querybuilder.NsToSeconds
}
if start != 0 {
sb.Where(sb.GE("timestamp", fmt.Sprintf("%d", start)), sb.GE("ts_bucket_start", startBucket))
}
if end != 0 {
sb.Where(sb.L("timestamp", fmt.Sprintf("%d", end)), sb.LE("ts_bucket_start", endBucket))
}
return preparedWhereClause, nil
}
func aggOrderBy(k qbtypes.OrderBy, q qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]) (int, bool) {
for i, agg := range q.Aggregations {
if k.Key.Name == agg.Alias || k.Key.Name == agg.Expression || k.Key.Name == fmt.Sprintf("%d", i) {
return i, true
}
}
return 0, false
}
func (b *auditQueryStatementBuilder) maybeAttachResourceFilter(
ctx context.Context,
sb *sqlbuilder.SelectBuilder,
query qbtypes.QueryBuilderQuery[qbtypes.LogAggregation],
start, end uint64,
variables map[string]qbtypes.VariableItem,
) (cteSQL string, cteArgs []any, err error) {
stmt, err := b.resourceFilterStmtBuilder.Build(ctx, start, end, qbtypes.RequestTypeRaw, query, variables)
if err != nil {
return "", nil, err
}
sb.Where("resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter)")
return fmt.Sprintf("__resource_filter AS (%s)", stmt.Query), stmt.Args, nil
}

View File

@@ -0,0 +1,241 @@
package telemetryaudit
import (
"context"
"testing"
"time"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/querybuilder/resourcefilter"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
"github.com/stretchr/testify/require"
)
func auditResourceFilterStmtBuilder() qbtypes.StatementBuilder[qbtypes.LogAggregation] {
fm := resourcefilter.NewFieldMapper()
cb := resourcefilter.NewConditionBuilder(fm)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = auditFieldKeyMap()
return resourcefilter.NewLogResourceFilterStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
fm,
cb,
mockMetadataStore,
DefaultFullTextColumn,
nil,
)
}
func auditFieldKeyMap() map[string][]*telemetrytypes.TelemetryFieldKey {
key := func(name string, ctx telemetrytypes.FieldContext, dt telemetrytypes.FieldDataType, materialized bool) *telemetrytypes.TelemetryFieldKey {
return &telemetrytypes.TelemetryFieldKey{
Name: name,
Signal: telemetrytypes.SignalLogs,
FieldContext: ctx,
FieldDataType: dt,
Materialized: materialized,
}
}
attr := telemetrytypes.FieldContextAttribute
res := telemetrytypes.FieldContextResource
str := telemetrytypes.FieldDataTypeString
i64 := telemetrytypes.FieldDataTypeInt64
return map[string][]*telemetrytypes.TelemetryFieldKey{
"service.name": {key("service.name", res, str, false)},
"signoz.audit.action": {key("signoz.audit.action", attr, str, true)},
"signoz.audit.outcome": {key("signoz.audit.outcome", attr, str, true)},
"signoz.audit.principal.email": {key("signoz.audit.principal.email", attr, str, true)},
"signoz.audit.principal.id": {key("signoz.audit.principal.id", attr, str, true)},
"signoz.audit.principal.type": {key("signoz.audit.principal.type", attr, str, true)},
"signoz.audit.resource.name": {key("signoz.audit.resource.name", attr, str, true)},
"signoz.audit.resource.id": {key("signoz.audit.resource.id", attr, str, true)},
"signoz.audit.action_category": {key("signoz.audit.action_category", attr, str, false)},
"signoz.audit.error.type": {key("signoz.audit.error.type", attr, str, false)},
"signoz.audit.error.code": {key("signoz.audit.error.code", attr, str, false)},
"http.request.method": {key("http.request.method", attr, str, false)},
"http.response.status_code": {key("http.response.status_code", attr, i64, false)},
}
}
func newTestAuditStatementBuilder() *auditQueryStatementBuilder {
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = auditFieldKeyMap()
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil)
return NewAuditQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
mockMetadataStore,
fm,
cb,
auditResourceFilterStmtBuilder(),
aggExprRewriter,
DefaultFullTextColumn,
nil,
)
}
func TestStatementBuilder(t *testing.T) {
statementBuilder := newTestAuditStatementBuilder()
ctx := context.Background()
testCases := []struct {
name string
requestType qbtypes.RequestType
query qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]
expected qbtypes.Statement
expectedErr error
}{
// List: all actions by a specific user (materialized principal.id filter)
{
name: "ListByPrincipalID",
requestType: qbtypes.RequestTypeRaw,
query: qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]{
Signal: telemetrytypes.SignalLogs,
Source: telemetrytypes.SourceAudit,
Filter: &qbtypes.Filter{
Expression: "signoz.audit.principal.id = '019a-1234-abcd-5678'",
},
Limit: 100,
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_logs.distributed_logs_v2_resource WHERE true AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ?) SELECT timestamp, id, trace_id, span_id, trace_flags, severity_text, severity_number, scope_name, scope_version, body, event_name, attributes_string, attributes_number, attributes_bool, scope_string FROM signoz_audit.distributed_logs WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND (`attribute_string_signoz$$audit$$principal$$id` = ? AND `attribute_string_signoz$$audit$$principal$$id_exists` = ?) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{uint64(1747945619), uint64(1747983448), "019a-1234-abcd-5678", true, "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 100},
},
},
// List: all failed actions (materialized outcome filter)
{
name: "ListByOutcomeFailure",
requestType: qbtypes.RequestTypeRaw,
query: qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]{
Signal: telemetrytypes.SignalLogs,
Source: telemetrytypes.SourceAudit,
Filter: &qbtypes.Filter{
Expression: "signoz.audit.outcome = 'failure'",
},
Limit: 100,
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_logs.distributed_logs_v2_resource WHERE true AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ?) SELECT timestamp, id, trace_id, span_id, trace_flags, severity_text, severity_number, scope_name, scope_version, body, event_name, attributes_string, attributes_number, attributes_bool, scope_string FROM signoz_audit.distributed_logs WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND (`attribute_string_signoz$$audit$$outcome` = ? AND `attribute_string_signoz$$audit$$outcome_exists` = ?) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{uint64(1747945619), uint64(1747983448), "failure", true, "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 100},
},
},
// List: change history of a specific dashboard (two materialized column AND)
{
name: "ListByResourceNameAndID",
requestType: qbtypes.RequestTypeRaw,
query: qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]{
Signal: telemetrytypes.SignalLogs,
Source: telemetrytypes.SourceAudit,
Filter: &qbtypes.Filter{
Expression: "signoz.audit.resource.name = 'dashboard' AND signoz.audit.resource.id = '019b-5678-efgh-9012'",
},
Limit: 100,
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_logs.distributed_logs_v2_resource WHERE true AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ?) SELECT timestamp, id, trace_id, span_id, trace_flags, severity_text, severity_number, scope_name, scope_version, body, event_name, attributes_string, attributes_number, attributes_bool, scope_string FROM signoz_audit.distributed_logs WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND ((`attribute_string_signoz$$audit$$resource$$name` = ? AND `attribute_string_signoz$$audit$$resource$$name_exists` = ?) AND (`attribute_string_signoz$$audit$$resource$$id` = ? AND `attribute_string_signoz$$audit$$resource$$id_exists` = ?)) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{uint64(1747945619), uint64(1747983448), "dashboard", true, "019b-5678-efgh-9012", true, "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 100},
},
},
// List: all dashboard deletions (compliance — resource.name + action AND)
{
name: "ListByResourceNameAndAction",
requestType: qbtypes.RequestTypeRaw,
query: qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]{
Signal: telemetrytypes.SignalLogs,
Source: telemetrytypes.SourceAudit,
Filter: &qbtypes.Filter{
Expression: "signoz.audit.resource.name = 'dashboard' AND signoz.audit.action = 'delete'",
},
Limit: 100,
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_logs.distributed_logs_v2_resource WHERE true AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ?) SELECT timestamp, id, trace_id, span_id, trace_flags, severity_text, severity_number, scope_name, scope_version, body, event_name, attributes_string, attributes_number, attributes_bool, scope_string FROM signoz_audit.distributed_logs WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND ((`attribute_string_signoz$$audit$$resource$$name` = ? AND `attribute_string_signoz$$audit$$resource$$name_exists` = ?) AND (`attribute_string_signoz$$audit$$action` = ? AND `attribute_string_signoz$$audit$$action_exists` = ?)) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{uint64(1747945619), uint64(1747983448), "dashboard", true, "delete", true, "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 100},
},
},
// List: all actions by service accounts (materialized principal.type)
{
name: "ListByPrincipalType",
requestType: qbtypes.RequestTypeRaw,
query: qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]{
Signal: telemetrytypes.SignalLogs,
Source: telemetrytypes.SourceAudit,
Filter: &qbtypes.Filter{
Expression: "signoz.audit.principal.type = 'service_account'",
},
Limit: 100,
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_logs.distributed_logs_v2_resource WHERE true AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ?) SELECT timestamp, id, trace_id, span_id, trace_flags, severity_text, severity_number, scope_name, scope_version, body, event_name, attributes_string, attributes_number, attributes_bool, scope_string FROM signoz_audit.distributed_logs WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND (`attribute_string_signoz$$audit$$principal$$type` = ? AND `attribute_string_signoz$$audit$$principal$$type_exists` = ?) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{uint64(1747945619), uint64(1747983448), "service_account", true, "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 100},
},
},
// Scalar: alert — count forbidden errors (outcome + action AND)
{
name: "ScalarCountByOutcomeAndAction",
requestType: qbtypes.RequestTypeScalar,
query: qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]{
Signal: telemetrytypes.SignalLogs,
Source: telemetrytypes.SourceAudit,
StepInterval: qbtypes.Step{Duration: 60 * time.Second},
Filter: &qbtypes.Filter{
Expression: "signoz.audit.outcome = 'failure' AND signoz.audit.action = 'update'",
},
Aggregations: []qbtypes.LogAggregation{
{Expression: "count()"},
},
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_logs.distributed_logs_v2_resource WHERE true AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ?) SELECT count() AS __result_0 FROM signoz_audit.distributed_logs WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND ((`attribute_string_signoz$$audit$$outcome` = ? AND `attribute_string_signoz$$audit$$outcome_exists` = ?) AND (`attribute_string_signoz$$audit$$action` = ? AND `attribute_string_signoz$$audit$$action_exists` = ?)) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? ORDER BY __result_0 DESC",
Args: []any{uint64(1747945619), uint64(1747983448), "failure", true, "update", true, "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448)},
},
},
// TimeSeries: failures grouped by principal email with top-N limit
{
name: "TimeSeriesFailuresGroupedByPrincipal",
requestType: qbtypes.RequestTypeTimeSeries,
query: qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]{
Signal: telemetrytypes.SignalLogs,
Source: telemetrytypes.SourceAudit,
StepInterval: qbtypes.Step{Duration: 60 * time.Second},
Aggregations: []qbtypes.LogAggregation{
{Expression: "count()"},
},
Filter: &qbtypes.Filter{
Expression: "signoz.audit.outcome = 'failure'",
},
GroupBy: []qbtypes.GroupByKey{
{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "signoz.audit.principal.email"}},
},
Limit: 5,
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_logs.distributed_logs_v2_resource WHERE true AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ?), __limit_cte AS (SELECT toString(multiIf(`attribute_string_signoz$$audit$$principal$$email_exists` = ?, `attribute_string_signoz$$audit$$principal$$email`, NULL)) AS `signoz.audit.principal.email`, count() AS __result_0 FROM signoz_audit.distributed_logs WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND (`attribute_string_signoz$$audit$$outcome` = ? AND `attribute_string_signoz$$audit$$outcome_exists` = ?) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? GROUP BY `signoz.audit.principal.email` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(fromUnixTimestamp64Nano(timestamp), INTERVAL 60 SECOND) AS ts, toString(multiIf(`attribute_string_signoz$$audit$$principal$$email_exists` = ?, `attribute_string_signoz$$audit$$principal$$email`, NULL)) AS `signoz.audit.principal.email`, count() AS __result_0 FROM signoz_audit.distributed_logs WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND (`attribute_string_signoz$$audit$$outcome` = ? AND `attribute_string_signoz$$audit$$outcome_exists` = ?) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? AND (`signoz.audit.principal.email`) GLOBAL IN (SELECT `signoz.audit.principal.email` FROM __limit_cte) GROUP BY ts, `signoz.audit.principal.email`",
Args: []any{uint64(1747945619), uint64(1747983448), true, "failure", true, "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 5, true, "failure", true, "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448)},
},
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
q, err := statementBuilder.Build(ctx, 1747947419000, 1747983448000, testCase.requestType, testCase.query, nil)
if testCase.expectedErr != nil {
require.Error(t, err)
require.Contains(t, err.Error(), testCase.expectedErr.Error())
} else {
require.NoError(t, err)
require.Equal(t, testCase.expected.Query, q.Query)
require.Equal(t, testCase.expected.Args, q.Args)
}
})
}
}

View File

@@ -0,0 +1,11 @@
package telemetryaudit
const (
DBName = "signoz_audit"
LogsTableName = "distributed_logs"
LogsLocalTableName = "logs"
TagAttributesTableName = "distributed_tag_attributes"
TagAttributesLocalTableName = "tag_attributes"
LogAttributeKeysTblName = "distributed_logs_attribute_keys"
LogResourceKeysTblName = "distributed_logs_resource_keys"
)

View File

@@ -13,6 +13,7 @@ import (
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetryaudit"
"github.com/SigNoz/signoz/pkg/telemetrylogs"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetrystore"
@@ -27,6 +28,7 @@ import (
var (
ErrFailedToGetTracesKeys = errors.Newf(errors.TypeInternal, errors.CodeInternal, "failed to get traces keys")
ErrFailedToGetLogsKeys = errors.Newf(errors.TypeInternal, errors.CodeInternal, "failed to get logs keys")
ErrFailedToGetAuditKeys = errors.Newf(errors.TypeInternal, errors.CodeInternal, "failed to get audit keys")
ErrFailedToGetTblStatement = errors.Newf(errors.TypeInternal, errors.CodeInternal, "failed to get tbl statement")
ErrFailedToGetMetricsKeys = errors.Newf(errors.TypeInternal, errors.CodeInternal, "failed to get metrics keys")
ErrFailedToGetMeterKeys = errors.Newf(errors.TypeInternal, errors.CodeInternal, "failed to get meter keys")
@@ -50,6 +52,11 @@ type telemetryMetaStore struct {
logAttributeKeysTblName string
logResourceKeysTblName string
logsV2TblName string
auditDBName string
auditV2TblName string
auditFieldsTblName string
auditAttributeKeysTblName string
auditResourceKeysTblName string
relatedMetadataDBName string
relatedMetadataTblName string
columnEvolutionMetadataTblName string
@@ -79,6 +86,11 @@ func NewTelemetryMetaStore(
logsFieldsTblName string,
logAttributeKeysTblName string,
logResourceKeysTblName string,
auditDBName string,
auditV2TblName string,
auditFieldsTblName string,
auditAttributeKeysTblName string,
auditResourceKeysTblName string,
relatedMetadataDBName string,
relatedMetadataTblName string,
columnEvolutionMetadataTblName string,
@@ -101,6 +113,11 @@ func NewTelemetryMetaStore(
logsFieldsTblName: logsFieldsTblName,
logAttributeKeysTblName: logAttributeKeysTblName,
logResourceKeysTblName: logResourceKeysTblName,
auditDBName: auditDBName,
auditV2TblName: auditV2TblName,
auditFieldsTblName: auditFieldsTblName,
auditAttributeKeysTblName: auditAttributeKeysTblName,
auditResourceKeysTblName: auditResourceKeysTblName,
relatedMetadataDBName: relatedMetadataDBName,
relatedMetadataTblName: relatedMetadataTblName,
columnEvolutionMetadataTblName: columnEvolutionMetadataTblName,
@@ -592,6 +609,240 @@ func (t *telemetryMetaStore) getLogsKeys(ctx context.Context, fieldKeySelectors
return keys, complete, nil
}
func (t *telemetryMetaStore) auditTblStatementToFieldKeys(ctx context.Context) ([]*telemetrytypes.TelemetryFieldKey, error) {
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.TelemetrySignal: telemetrytypes.SignalLogs.StringValue(),
instrumentationtypes.CodeNamespace: "metadata",
instrumentationtypes.CodeFunctionName: "auditTblStatementToFieldKeys",
})
if t.auditDBName == "" || t.auditV2TblName == "" {
return nil, nil
}
query := fmt.Sprintf("SHOW CREATE TABLE %s.%s", t.auditDBName, t.auditV2TblName)
statements := []telemetrytypes.ShowCreateTableStatement{}
err := t.telemetrystore.ClickhouseDB().Select(ctx, &statements, query)
if err != nil {
return nil, errors.Wrap(err, errors.TypeInternal, errors.CodeInternal, ErrFailedToGetTblStatement.Error())
}
materialisedKeys, err := ExtractFieldKeysFromTblStatement(statements[0].Statement)
if err != nil {
return nil, errors.Wrap(err, errors.TypeInternal, errors.CodeInternal, ErrFailedToGetAuditKeys.Error())
}
for idx := range materialisedKeys {
materialisedKeys[idx].Signal = telemetrytypes.SignalLogs
}
return materialisedKeys, nil
}
func (t *telemetryMetaStore) getAuditKeys(ctx context.Context, fieldKeySelectors []*telemetrytypes.FieldKeySelector) ([]*telemetrytypes.TelemetryFieldKey, bool, error) {
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.TelemetrySignal: telemetrytypes.SignalLogs.StringValue(),
instrumentationtypes.CodeNamespace: "metadata",
instrumentationtypes.CodeFunctionName: "getAuditKeys",
})
if len(fieldKeySelectors) == 0 {
return nil, true, nil
}
if t.auditDBName == "" {
return nil, true, nil
}
matKeys, err := t.auditTblStatementToFieldKeys(ctx)
if err != nil {
return nil, false, err
}
mapOfKeys := make(map[string]*telemetrytypes.TelemetryFieldKey)
for _, key := range matKeys {
mapOfKeys[key.Name+";"+key.FieldContext.StringValue()+";"+key.FieldDataType.StringValue()] = key
}
var queries []string
var allArgs []any
queryAttributeTable := false
queryResourceTable := false
for _, selector := range fieldKeySelectors {
if selector.FieldContext == telemetrytypes.FieldContextUnspecified {
queryAttributeTable = true
queryResourceTable = true
break
} else if selector.FieldContext == telemetrytypes.FieldContextAttribute {
queryAttributeTable = true
} else if selector.FieldContext == telemetrytypes.FieldContextResource {
queryResourceTable = true
}
}
tablesToQuery := []struct {
fieldContext telemetrytypes.FieldContext
shouldQuery bool
}{
{telemetrytypes.FieldContextAttribute, queryAttributeTable},
{telemetrytypes.FieldContextResource, queryResourceTable},
}
for _, table := range tablesToQuery {
if !table.shouldQuery {
continue
}
fieldContext := table.fieldContext
var tblName string
if fieldContext == telemetrytypes.FieldContextAttribute {
tblName = t.auditDBName + "." + t.auditAttributeKeysTblName
} else {
tblName = t.auditDBName + "." + t.auditResourceKeysTblName
}
sb := sqlbuilder.Select(
"name AS tag_key",
fmt.Sprintf("'%s' AS tag_type", fieldContext.TagType()),
"lower(datatype) AS tag_data_type",
fmt.Sprintf("%d AS priority", getPriorityForContext(fieldContext)),
).From(tblName)
var limit int
conds := []string{}
for _, fieldKeySelector := range fieldKeySelectors {
if fieldKeySelector.FieldContext != telemetrytypes.FieldContextUnspecified && fieldKeySelector.FieldContext != fieldContext {
continue
}
fieldKeyConds := []string{}
if fieldKeySelector.SelectorMatchType == telemetrytypes.FieldSelectorMatchTypeExact {
fieldKeyConds = append(fieldKeyConds, sb.E("name", fieldKeySelector.Name))
} else {
fieldKeyConds = append(fieldKeyConds, sb.ILike("name", "%"+escapeForLike(fieldKeySelector.Name)+"%"))
}
if fieldKeySelector.FieldDataType != telemetrytypes.FieldDataTypeUnspecified {
fieldKeyConds = append(fieldKeyConds, sb.E("datatype", fieldKeySelector.FieldDataType.TagDataType()))
}
if len(fieldKeyConds) > 0 {
conds = append(conds, sb.And(fieldKeyConds...))
}
limit += fieldKeySelector.Limit
}
if len(conds) > 0 {
sb.Where(sb.Or(conds...))
}
sb.GroupBy("name", "datatype")
if limit == 0 {
limit = 1000
}
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
queries = append(queries, query)
allArgs = append(allArgs, args...)
}
if len(queries) == 0 {
return []*telemetrytypes.TelemetryFieldKey{}, true, nil
}
var limit int
for _, fieldKeySelector := range fieldKeySelectors {
limit += fieldKeySelector.Limit
}
if limit == 0 {
limit = 1000
}
mainQuery := fmt.Sprintf(`
SELECT tag_key, tag_type, tag_data_type, max(priority) as priority
FROM (
%s
) AS combined_results
GROUP BY tag_key, tag_type, tag_data_type
ORDER BY priority
LIMIT %d
`, strings.Join(queries, " UNION ALL "), limit+1)
rows, err := t.telemetrystore.ClickhouseDB().Query(ctx, mainQuery, allArgs...)
if err != nil {
return nil, false, errors.Wrap(err, errors.TypeInternal, errors.CodeInternal, ErrFailedToGetAuditKeys.Error())
}
defer rows.Close()
keys := []*telemetrytypes.TelemetryFieldKey{}
rowCount := 0
searchTexts := []string{}
for _, fieldKeySelector := range fieldKeySelectors {
searchTexts = append(searchTexts, fieldKeySelector.Name)
}
for rows.Next() {
rowCount++
if rowCount > limit {
break
}
var name string
var fieldContext telemetrytypes.FieldContext
var fieldDataType telemetrytypes.FieldDataType
var priority uint8
err = rows.Scan(&name, &fieldContext, &fieldDataType, &priority)
if err != nil {
return nil, false, errors.Wrap(err, errors.TypeInternal, errors.CodeInternal, ErrFailedToGetAuditKeys.Error())
}
key, ok := mapOfKeys[name+";"+fieldContext.StringValue()+";"+fieldDataType.StringValue()]
if !ok {
key = &telemetrytypes.TelemetryFieldKey{
Name: name,
Signal: telemetrytypes.SignalLogs,
FieldContext: fieldContext,
FieldDataType: fieldDataType,
}
}
keys = append(keys, key)
mapOfKeys[name+";"+fieldContext.StringValue()+";"+fieldDataType.StringValue()] = key
}
if rows.Err() != nil {
return nil, false, errors.Wrap(rows.Err(), errors.TypeInternal, errors.CodeInternal, ErrFailedToGetAuditKeys.Error())
}
complete := rowCount <= limit
// Add intrinsic audit fields (same as logs intrinsics: body, severity_text, etc.)
staticKeys := maps.Keys(telemetryaudit.IntrinsicFields)
for _, key := range staticKeys {
found := false
for _, v := range searchTexts {
if v == "" || strings.Contains(key, v) {
found = true
break
}
}
if found {
if field, exists := telemetryaudit.IntrinsicFields[key]; exists {
if _, added := mapOfKeys[field.Name+";"+field.FieldContext.StringValue()+";"+field.FieldDataType.StringValue()]; !added {
keys = append(keys, &field)
}
}
}
}
return keys, complete, nil
}
func getPriorityForContext(ctx telemetrytypes.FieldContext) int {
switch ctx {
case telemetrytypes.FieldContextLog:
@@ -889,7 +1140,11 @@ func (t *telemetryMetaStore) GetKeys(ctx context.Context, fieldKeySelector *tele
case telemetrytypes.SignalTraces:
keys, complete, err = t.getTracesKeys(ctx, selectors)
case telemetrytypes.SignalLogs:
keys, complete, err = t.getLogsKeys(ctx, selectors)
if fieldKeySelector.Source == telemetrytypes.SourceAudit {
keys, complete, err = t.getAuditKeys(ctx, selectors)
} else {
keys, complete, err = t.getLogsKeys(ctx, selectors)
}
case telemetrytypes.SignalMetrics:
if fieldKeySelector.Source == telemetrytypes.SourceMeter {
keys, complete, err = t.getMeterSourceMetricKeys(ctx, selectors)
@@ -938,6 +1193,7 @@ func (t *telemetryMetaStore) GetKeys(ctx context.Context, fieldKeySelector *tele
func (t *telemetryMetaStore) GetKeysMulti(ctx context.Context, fieldKeySelectors []*telemetrytypes.FieldKeySelector) (map[string][]*telemetrytypes.TelemetryFieldKey, bool, error) {
logsSelectors := []*telemetrytypes.FieldKeySelector{}
auditSelectors := []*telemetrytypes.FieldKeySelector{}
tracesSelectors := []*telemetrytypes.FieldKeySelector{}
metricsSelectors := []*telemetrytypes.FieldKeySelector{}
meterSourceMetricsSelectors := []*telemetrytypes.FieldKeySelector{}
@@ -945,7 +1201,11 @@ func (t *telemetryMetaStore) GetKeysMulti(ctx context.Context, fieldKeySelectors
for _, fieldKeySelector := range fieldKeySelectors {
switch fieldKeySelector.Signal {
case telemetrytypes.SignalLogs:
logsSelectors = append(logsSelectors, fieldKeySelector)
if fieldKeySelector.Source == telemetrytypes.SourceAudit {
auditSelectors = append(auditSelectors, fieldKeySelector)
} else {
logsSelectors = append(logsSelectors, fieldKeySelector)
}
case telemetrytypes.SignalTraces:
tracesSelectors = append(tracesSelectors, fieldKeySelector)
case telemetrytypes.SignalMetrics:
@@ -965,6 +1225,10 @@ func (t *telemetryMetaStore) GetKeysMulti(ctx context.Context, fieldKeySelectors
if err != nil {
return nil, false, err
}
auditKeys, auditComplete, err := t.getAuditKeys(ctx, auditSelectors)
if err != nil {
return nil, false, err
}
tracesKeys, tracesComplete, err := t.getTracesKeys(ctx, tracesSelectors)
if err != nil {
return nil, false, err
@@ -979,12 +1243,15 @@ func (t *telemetryMetaStore) GetKeysMulti(ctx context.Context, fieldKeySelectors
return nil, false, err
}
// Complete only if all queries are complete
complete := logsComplete && tracesComplete && metricsComplete
complete := logsComplete && auditComplete && tracesComplete && metricsComplete
mapOfKeys := make(map[string][]*telemetrytypes.TelemetryFieldKey)
for _, key := range logsKeys {
mapOfKeys[key.Name] = append(mapOfKeys[key.Name], key)
}
for _, key := range auditKeys {
mapOfKeys[key.Name] = append(mapOfKeys[key.Name], key)
}
for _, key := range tracesKeys {
mapOfKeys[key.Name] = append(mapOfKeys[key.Name], key)
}
@@ -1338,6 +1605,100 @@ func (t *telemetryMetaStore) getLogFieldValues(ctx context.Context, fieldValueSe
return values, complete, nil
}
func (t *telemetryMetaStore) getAuditFieldValues(ctx context.Context, fieldValueSelector *telemetrytypes.FieldValueSelector) (*telemetrytypes.TelemetryFieldValues, bool, error) {
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.TelemetrySignal: telemetrytypes.SignalLogs.StringValue(),
instrumentationtypes.CodeNamespace: "metadata",
instrumentationtypes.CodeFunctionName: "getAuditFieldValues",
})
if t.auditDBName == "" || t.auditFieldsTblName == "" {
return &telemetrytypes.TelemetryFieldValues{}, true, nil
}
limit := fieldValueSelector.Limit
if limit == 0 {
limit = 50
}
sb := sqlbuilder.Select("DISTINCT string_value, number_value").From(t.auditDBName + "." + t.auditFieldsTblName)
if fieldValueSelector.Name != "" {
sb.Where(sb.E("tag_key", fieldValueSelector.Name))
}
if fieldValueSelector.FieldContext != telemetrytypes.FieldContextUnspecified {
sb.Where(sb.E("tag_type", fieldValueSelector.FieldContext.TagType()))
}
if fieldValueSelector.FieldDataType != telemetrytypes.FieldDataTypeUnspecified {
sb.Where(sb.E("tag_data_type", fieldValueSelector.FieldDataType.TagDataType()))
}
if fieldValueSelector.Value != "" {
switch fieldValueSelector.FieldDataType {
case telemetrytypes.FieldDataTypeString:
sb.Where(sb.ILike("string_value", "%"+escapeForLike(fieldValueSelector.Value)+"%"))
case telemetrytypes.FieldDataTypeNumber:
sb.Where(sb.IsNotNull("number_value"))
sb.Where(sb.ILike("toString(number_value)", "%"+escapeForLike(fieldValueSelector.Value)+"%"))
case telemetrytypes.FieldDataTypeUnspecified:
sb.Where(sb.Or(
sb.ILike("string_value", "%"+escapeForLike(fieldValueSelector.Value)+"%"),
sb.ILike("toString(number_value)", "%"+escapeForLike(fieldValueSelector.Value)+"%"),
))
}
}
sb.Limit(limit + 1)
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
rows, err := t.telemetrystore.ClickhouseDB().Query(ctx, query, args...)
if err != nil {
return nil, false, errors.Wrap(err, errors.TypeInternal, errors.CodeInternal, ErrFailedToGetAuditKeys.Error())
}
defer rows.Close()
values := &telemetrytypes.TelemetryFieldValues{}
seen := make(map[string]bool)
rowCount := 0
totalCount := 0
for rows.Next() {
rowCount++
var stringValue string
var numberValue float64
err = rows.Scan(&stringValue, &numberValue)
if err != nil {
return nil, false, errors.Wrap(err, errors.TypeInternal, errors.CodeInternal, ErrFailedToGetAuditKeys.Error())
}
if stringValue != "" && !seen[stringValue] {
if totalCount >= limit {
break
}
values.StringValues = append(values.StringValues, stringValue)
seen[stringValue] = true
totalCount++
}
if numberValue != 0 {
if totalCount >= limit {
break
}
if !seen[fmt.Sprintf("%f", numberValue)] {
values.NumberValues = append(values.NumberValues, numberValue)
seen[fmt.Sprintf("%f", numberValue)] = true
totalCount++
}
}
}
complete := rowCount <= limit
return values, complete, nil
}
// getMetricFieldValues returns field values and whether the result is complete.
func (t *telemetryMetaStore) getMetricFieldValues(ctx context.Context, fieldValueSelector *telemetrytypes.FieldValueSelector) (*telemetrytypes.TelemetryFieldValues, bool, error) {
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
@@ -1628,7 +1989,11 @@ func (t *telemetryMetaStore) GetAllValues(ctx context.Context, fieldValueSelecto
case telemetrytypes.SignalTraces:
values, complete, err = t.getSpanFieldValues(ctx, fieldValueSelector)
case telemetrytypes.SignalLogs:
values, complete, err = t.getLogFieldValues(ctx, fieldValueSelector)
if fieldValueSelector.Source == telemetrytypes.SourceAudit {
values, complete, err = t.getAuditFieldValues(ctx, fieldValueSelector)
} else {
values, complete, err = t.getLogFieldValues(ctx, fieldValueSelector)
}
case telemetrytypes.SignalMetrics:
if fieldValueSelector.Source == telemetrytypes.SourceMeter {
values, complete, err = t.getMeterSourceMetricFieldValues(ctx, fieldValueSelector)

View File

@@ -5,6 +5,7 @@ import (
"testing"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/telemetryaudit"
"github.com/SigNoz/signoz/pkg/telemetrylogs"
"github.com/SigNoz/signoz/pkg/telemetrymeter"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
@@ -37,6 +38,11 @@ func TestGetFirstSeenFromMetricMetadata(t *testing.T) {
telemetrylogs.TagAttributesV2TableName,
telemetrylogs.LogAttributeKeysTblName,
telemetrylogs.LogResourceKeysTblName,
telemetryaudit.DBName,
telemetryaudit.LogsTableName,
telemetryaudit.TagAttributesTableName,
telemetryaudit.LogAttributeKeysTblName,
telemetryaudit.LogResourceKeysTblName,
DBName,
AttributesMetadataLocalTableName,
ColumnEvolutionMetadataTableName,

View File

@@ -7,6 +7,7 @@ import (
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/telemetryaudit"
"github.com/SigNoz/signoz/pkg/telemetrylogs"
"github.com/SigNoz/signoz/pkg/telemetrymeter"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
@@ -36,6 +37,11 @@ func newTestTelemetryMetaStoreTestHelper(store telemetrystore.TelemetryStore) te
telemetrylogs.TagAttributesV2TableName,
telemetrylogs.LogAttributeKeysTblName,
telemetrylogs.LogResourceKeysTblName,
telemetryaudit.DBName,
telemetryaudit.LogsTableName,
telemetryaudit.TagAttributesTableName,
telemetryaudit.LogAttributeKeysTblName,
telemetryaudit.LogResourceKeysTblName,
DBName,
AttributesMetadataLocalTableName,
ColumnEvolutionMetadataTableName,

View File

@@ -7,11 +7,13 @@ type Source struct {
}
var (
SourceAudit = Source{valuer.NewString("audit")}
SourceMeter = Source{valuer.NewString("meter")}
SourceUnspecified = Source{valuer.NewString("")}
)
// Enum returns the acceptable values for Source.
// TODO: Add SourceAudit once the frontend is ready for consumption.
func (Source) Enum() []any {
return []any{
SourceMeter,

View File

@@ -12,6 +12,7 @@ pytest_plugins = [
"fixtures.sqlite",
"fixtures.zookeeper",
"fixtures.signoz",
"fixtures.audit",
"fixtures.logs",
"fixtures.traces",
"fixtures.metrics",
@@ -50,12 +51,6 @@ def pytest_addoption(parser: pytest.Parser):
default="postgres",
help="sqlstore provider",
)
parser.addoption(
"--sqlite-mode",
action="store",
default="delete",
help="sqlite mode",
)
parser.addoption(
"--postgres-version",
action="store",

View File

@@ -0,0 +1,404 @@
import datetime
import json
from abc import ABC
from typing import Any, Callable, Generator, List, Optional
import numpy as np
import pytest
from ksuid import KsuidMs
from fixtures import types
from fixtures.fingerprint import LogsOrTracesFingerprint
class AuditResource(ABC):
labels: str
fingerprint: str
seen_at_ts_bucket_start: np.int64
def __init__(
self,
labels: dict[str, str],
fingerprint: str,
seen_at_ts_bucket_start: np.int64,
) -> None:
self.labels = json.dumps(labels, separators=(",", ":"))
self.fingerprint = fingerprint
self.seen_at_ts_bucket_start = seen_at_ts_bucket_start
def np_arr(self) -> np.array:
return np.array(
[
self.labels,
self.fingerprint,
self.seen_at_ts_bucket_start,
]
)
class AuditResourceOrAttributeKeys(ABC):
name: str
datatype: str
def __init__(self, name: str, datatype: str) -> None:
self.name = name
self.datatype = datatype
def np_arr(self) -> np.array:
return np.array([self.name, self.datatype])
class AuditTagAttributes(ABC):
unix_milli: np.int64
tag_key: str
tag_type: str
tag_data_type: str
string_value: str
int64_value: Optional[np.int64]
float64_value: Optional[np.float64]
def __init__(
self,
timestamp: datetime.datetime,
tag_key: str,
tag_type: str,
tag_data_type: str,
string_value: Optional[str],
int64_value: Optional[np.int64],
float64_value: Optional[np.float64],
) -> None:
self.unix_milli = np.int64(int(timestamp.timestamp() * 1e3))
self.tag_key = tag_key
self.tag_type = tag_type
self.tag_data_type = tag_data_type
self.string_value = string_value or ""
self.int64_value = int64_value
self.float64_value = float64_value
def np_arr(self) -> np.array:
return np.array(
[
self.unix_milli,
self.tag_key,
self.tag_type,
self.tag_data_type,
self.string_value,
self.int64_value,
self.float64_value,
]
)
class AuditLog(ABC):
"""Represents a single audit log event in signoz_audit.
Matches the ClickHouse DDL from the schema migration (ticket #1936):
- Database: signoz_audit
- Local table: logs
- Distributed table: distributed_logs
- No resources_string column (resource JSON only)
- Has event_name column
- 7 materialized columns auto-populated from attributes_string at INSERT time
"""
ts_bucket_start: np.uint64
resource_fingerprint: str
timestamp: np.uint64
observed_timestamp: np.uint64
id: str
trace_id: str
span_id: str
trace_flags: np.uint32
severity_text: str
severity_number: np.uint8
body: str
scope_name: str
scope_version: str
scope_string: dict[str, str]
attributes_string: dict[str, str]
attributes_number: dict[str, np.float64]
attributes_bool: dict[str, bool]
resource_json: dict[str, str]
event_name: str
resource: List[AuditResource]
tag_attributes: List[AuditTagAttributes]
resource_keys: List[AuditResourceOrAttributeKeys]
attribute_keys: List[AuditResourceOrAttributeKeys]
def __init__(
self,
timestamp: Optional[datetime.datetime] = None,
resources: dict[str, Any] = {},
attributes: dict[str, Any] = {},
body: str = "",
event_name: str = "",
severity_text: str = "INFO",
trace_id: str = "",
span_id: str = "",
trace_flags: np.uint32 = 0,
scope_name: str = "signoz.audit",
scope_version: str = "",
) -> None:
if timestamp is None:
timestamp = datetime.datetime.now()
self.tag_attributes = []
self.attribute_keys = []
self.resource_keys = []
self.timestamp = np.uint64(int(timestamp.timestamp() * 1e9))
self.observed_timestamp = self.timestamp
minute = timestamp.minute
bucket_minute = 0 if minute < 30 else 30
bucket_start = timestamp.replace(minute=bucket_minute, second=0, microsecond=0)
self.ts_bucket_start = np.uint64(int(bucket_start.timestamp()))
self.id = str(KsuidMs(datetime=timestamp))
self.trace_id = trace_id
self.span_id = span_id
self.trace_flags = trace_flags
self.severity_text = severity_text
self.severity_number = np.uint8(9 if severity_text == "INFO" else 17)
self.body = body
self.event_name = event_name
# Resources — JSON column only (no resources_string in audit DDL)
self.resource_json = {k: str(v) for k, v in resources.items()}
for k, v in self.resource_json.items():
self.tag_attributes.append(
AuditTagAttributes(
timestamp=timestamp,
tag_key=k,
tag_type="resource",
tag_data_type="string",
string_value=str(v),
int64_value=None,
float64_value=None,
)
)
self.resource_keys.append(
AuditResourceOrAttributeKeys(name=k, datatype="string")
)
self.resource_fingerprint = LogsOrTracesFingerprint(
self.resource_json
).calculate()
# Process attributes by type
self.attributes_string = {}
self.attributes_number = {}
self.attributes_bool = {}
for k, v in attributes.items():
if isinstance(v, bool):
self.attributes_bool[k] = v
self.tag_attributes.append(
AuditTagAttributes(
timestamp=timestamp,
tag_key=k,
tag_type="tag",
tag_data_type="bool",
string_value=None,
int64_value=None,
float64_value=None,
)
)
self.attribute_keys.append(
AuditResourceOrAttributeKeys(name=k, datatype="bool")
)
elif isinstance(v, int):
self.attributes_number[k] = np.float64(v)
self.tag_attributes.append(
AuditTagAttributes(
timestamp=timestamp,
tag_key=k,
tag_type="tag",
tag_data_type="int64",
string_value=None,
int64_value=np.int64(v),
float64_value=None,
)
)
self.attribute_keys.append(
AuditResourceOrAttributeKeys(name=k, datatype="int64")
)
elif isinstance(v, float):
self.attributes_number[k] = np.float64(v)
self.tag_attributes.append(
AuditTagAttributes(
timestamp=timestamp,
tag_key=k,
tag_type="tag",
tag_data_type="float64",
string_value=None,
int64_value=None,
float64_value=np.float64(v),
)
)
self.attribute_keys.append(
AuditResourceOrAttributeKeys(name=k, datatype="float64")
)
else:
self.attributes_string[k] = str(v)
self.tag_attributes.append(
AuditTagAttributes(
timestamp=timestamp,
tag_key=k,
tag_type="tag",
tag_data_type="string",
string_value=str(v),
int64_value=None,
float64_value=None,
)
)
self.attribute_keys.append(
AuditResourceOrAttributeKeys(name=k, datatype="string")
)
self.scope_name = scope_name
self.scope_version = scope_version
self.scope_string = {}
self.resource = [
AuditResource(
labels=self.resource_json,
fingerprint=self.resource_fingerprint,
seen_at_ts_bucket_start=self.ts_bucket_start,
)
]
def np_arr(self) -> np.array:
return np.array(
[
self.ts_bucket_start,
self.resource_fingerprint,
self.timestamp,
self.observed_timestamp,
self.id,
self.trace_id,
self.span_id,
self.trace_flags,
self.severity_text,
self.severity_number,
self.body,
self.scope_name,
self.scope_version,
self.scope_string,
self.attributes_string,
self.attributes_number,
self.attributes_bool,
self.resource_json,
self.event_name,
]
)
@pytest.fixture(name="insert_audit_logs", scope="function")
def insert_audit_logs(
clickhouse: types.TestContainerClickhouse,
) -> Generator[Callable[[List[AuditLog]], None], Any, None]:
def _insert_audit_logs(logs: List[AuditLog]) -> None:
resources: List[AuditResource] = []
for log in logs:
resources.extend(log.resource)
if len(resources) > 0:
clickhouse.conn.insert(
database="signoz_audit",
table="distributed_logs_resource",
data=[resource.np_arr() for resource in resources],
column_names=[
"labels",
"fingerprint",
"seen_at_ts_bucket_start",
],
)
tag_attributes: List[AuditTagAttributes] = []
for log in logs:
tag_attributes.extend(log.tag_attributes)
if len(tag_attributes) > 0:
clickhouse.conn.insert(
database="signoz_audit",
table="distributed_tag_attributes",
data=[ta.np_arr() for ta in tag_attributes],
column_names=[
"unix_milli",
"tag_key",
"tag_type",
"tag_data_type",
"string_value",
"int64_value",
"float64_value",
],
)
attribute_keys: List[AuditResourceOrAttributeKeys] = []
for log in logs:
attribute_keys.extend(log.attribute_keys)
if len(attribute_keys) > 0:
clickhouse.conn.insert(
database="signoz_audit",
table="distributed_logs_attribute_keys",
data=[ak.np_arr() for ak in attribute_keys],
column_names=["name", "datatype"],
)
resource_keys: List[AuditResourceOrAttributeKeys] = []
for log in logs:
resource_keys.extend(log.resource_keys)
if len(resource_keys) > 0:
clickhouse.conn.insert(
database="signoz_audit",
table="distributed_logs_resource_keys",
data=[rk.np_arr() for rk in resource_keys],
column_names=["name", "datatype"],
)
clickhouse.conn.insert(
database="signoz_audit",
table="distributed_logs",
data=[log.np_arr() for log in logs],
column_names=[
"ts_bucket_start",
"resource_fingerprint",
"timestamp",
"observed_timestamp",
"id",
"trace_id",
"span_id",
"trace_flags",
"severity_text",
"severity_number",
"body",
"scope_name",
"scope_version",
"scope_string",
"attributes_string",
"attributes_number",
"attributes_bool",
"resource",
"event_name",
],
)
yield _insert_audit_logs
cluster = clickhouse.env["SIGNOZ_TELEMETRYSTORE_CLICKHOUSE_CLUSTER"]
for table in [
"logs",
"logs_resource",
"tag_attributes",
"logs_attribute_keys",
"logs_resource_keys",
]:
clickhouse.conn.query(
f"TRUNCATE TABLE signoz_audit.{table} ON CLUSTER '{cluster}' SYNC"
)

View File

@@ -28,7 +28,6 @@ def sqlite(
result = conn.execute(sql.text("SELECT 1"))
assert result.fetchone()[0] == 1
mode = pytestconfig.getoption("--sqlite-mode")
return types.TestContainerSQL(
container=types.TestContainerDocker(
id="",
@@ -39,7 +38,6 @@ def sqlite(
env={
"SIGNOZ_SQLSTORE_PROVIDER": "sqlite",
"SIGNOZ_SQLSTORE_SQLITE_PATH": str(path),
"SIGNOZ_SQLSTORE_SQLITE_MODE": mode,
},
)

View File

@@ -0,0 +1,414 @@
"""Integration tests for audit log querying via /api/v5/query_range.
Tests verify that audit events inserted directly into signoz_audit ClickHouse
tables can be queried back through the standard query_range API with
signal=logs, source=audit.
Each test maps to a real user query pattern from the audit logs design doc.
"""
from datetime import datetime, timedelta, timezone
from http import HTTPStatus
from typing import Callable, List
import requests
from fixtures import types
from fixtures.audit import AuditLog
from fixtures.auth import USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD
from fixtures.querier import make_query_request
def _build_audit_query(
*,
filter_expression: str = "",
limit: int = 100,
source: str = "audit",
) -> dict:
spec = {
"name": "A",
"signal": "logs",
"source": source,
"disabled": False,
"limit": limit,
"offset": 0,
"order": [
{"key": {"name": "timestamp"}, "direction": "desc"},
{"key": {"name": "id"}, "direction": "desc"},
],
"aggregations": [{"expression": "count()"}],
}
if filter_expression:
spec["filter"] = {"expression": filter_expression}
return {"type": "builder_query", "spec": spec}
def _build_audit_ts_query(
*,
aggregation: str = "count()",
filter_expression: str = "",
group_by: str = "",
step_interval: int = 60,
limit: int = 0,
) -> dict:
spec = {
"name": "A",
"signal": "logs",
"source": "audit",
"stepInterval": step_interval,
"aggregations": [{"expression": aggregation}],
}
if filter_expression:
spec["filter"] = {"expression": filter_expression}
if group_by:
spec["groupBy"] = [{"name": group_by}]
if limit:
spec["limit"] = limit
return {"type": "builder_query", "spec": spec}
def _query_audit_raw(
signoz: types.SigNoz,
token: str,
query: dict,
) -> requests.Response:
now = datetime.now(tz=timezone.utc)
return make_query_request(
signoz,
token,
start_ms=int((now - timedelta(seconds=30)).timestamp() * 1000),
end_ms=int(now.timestamp() * 1000),
queries=[query],
request_type="raw",
)
def _query_audit_ts(
signoz: types.SigNoz,
token: str,
query: dict,
) -> requests.Response:
now = datetime.now(tz=timezone.utc)
return make_query_request(
signoz,
token,
start_ms=int((now - timedelta(minutes=5)).timestamp() * 1000),
end_ms=int(now.timestamp() * 1000),
queries=[query],
request_type="time_series",
)
def _insert_standard_audit_events(
insert_audit_logs: Callable[[List[AuditLog]], None],
) -> None:
"""Insert a representative set of audit events for testing."""
now = datetime.now(tz=timezone.utc)
insert_audit_logs(
[
# Success: admin creates a dashboard
AuditLog(
timestamp=now - timedelta(seconds=5),
resources={"service.name": "signoz", "service.version": "0.90.0"},
attributes={
"signoz.audit.principal.id": "user-001",
"signoz.audit.principal.email": "alice@acme.com",
"signoz.audit.principal.type": "user",
"signoz.audit.principal.org_id": "org-001",
"signoz.audit.action": "create",
"signoz.audit.action_category": "configuration_change",
"signoz.audit.outcome": "success",
"signoz.audit.resource.name": "dashboard",
"signoz.audit.resource.id": "dash-001",
},
body="alice@acme.com (user-001) created dashboard (dash-001)",
event_name="dashboard.created",
severity_text="INFO",
scope_name="signoz.audit",
),
# Success: admin updates a dashboard
AuditLog(
timestamp=now - timedelta(seconds=4),
resources={"service.name": "signoz", "service.version": "0.90.0"},
attributes={
"signoz.audit.principal.id": "user-001",
"signoz.audit.principal.email": "alice@acme.com",
"signoz.audit.principal.type": "user",
"signoz.audit.principal.org_id": "org-001",
"signoz.audit.action": "update",
"signoz.audit.action_category": "configuration_change",
"signoz.audit.outcome": "success",
"signoz.audit.resource.name": "dashboard",
"signoz.audit.resource.id": "dash-001",
},
body="alice@acme.com (user-001) updated dashboard (dash-001)",
event_name="dashboard.updated",
severity_text="INFO",
scope_name="signoz.audit",
),
# Failure: viewer tries to delete a dashboard
AuditLog(
timestamp=now - timedelta(seconds=3),
resources={"service.name": "signoz", "service.version": "0.90.0"},
attributes={
"signoz.audit.principal.id": "user-002",
"signoz.audit.principal.email": "viewer@acme.com",
"signoz.audit.principal.type": "user",
"signoz.audit.principal.org_id": "org-001",
"signoz.audit.action": "delete",
"signoz.audit.action_category": "configuration_change",
"signoz.audit.outcome": "failure",
"signoz.audit.resource.name": "dashboard",
"signoz.audit.resource.id": "dash-001",
"signoz.audit.error.type": "forbidden",
"signoz.audit.error.code": "authz_forbidden",
},
body="viewer@acme.com (user-002) failed to delete dashboard (dash-001): forbidden (authz_forbidden)",
event_name="dashboard.deleted",
severity_text="ERROR",
scope_name="signoz.audit",
),
# Success: service account creates an API key
AuditLog(
timestamp=now - timedelta(seconds=2),
resources={"service.name": "signoz", "service.version": "0.90.0"},
attributes={
"signoz.audit.principal.id": "sa-001",
"signoz.audit.principal.email": "",
"signoz.audit.principal.type": "service_account",
"signoz.audit.principal.org_id": "org-001",
"signoz.audit.action": "create",
"signoz.audit.action_category": "access_control",
"signoz.audit.outcome": "success",
"signoz.audit.resource.name": "serviceaccount",
"signoz.audit.resource.id": "sa-001",
},
body="sa-001 created serviceaccount (sa-001)",
event_name="serviceaccount.apikey.created",
severity_text="INFO",
scope_name="signoz.audit",
),
# Success: admin logs in
AuditLog(
timestamp=now - timedelta(seconds=1),
resources={"service.name": "signoz", "service.version": "0.90.0"},
attributes={
"signoz.audit.principal.id": "user-001",
"signoz.audit.principal.email": "alice@acme.com",
"signoz.audit.principal.type": "user",
"signoz.audit.principal.org_id": "org-001",
"signoz.audit.action": "login",
"signoz.audit.action_category": "access_control",
"signoz.audit.outcome": "success",
"signoz.audit.resource.name": "session",
"signoz.audit.resource.id": "*",
},
body="alice@acme.com (user-001) login session (*)",
event_name="session.login",
severity_text="INFO",
scope_name="signoz.audit",
),
]
)
def test_audit_list_all(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_audit_logs: Callable[[List[AuditLog]], None],
) -> None:
"""List all audit events — verify correct count and ordering."""
_insert_standard_audit_events(insert_audit_logs)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = _query_audit_raw(signoz, token, _build_audit_query())
assert response.status_code == HTTPStatus.OK
assert response.json()["status"] == "success"
results = response.json()["data"]["data"]["results"]
assert len(results) == 1
rows = results[0]["rows"]
assert len(rows) == 5
# Most recent first (session.login)
assert rows[0]["data"]["event_name"] == "session.login"
# Oldest last (dashboard.created)
assert rows[4]["data"]["event_name"] == "dashboard.created"
def test_audit_filter_by_principal(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_audit_logs: Callable[[List[AuditLog]], None],
) -> None:
"""Q1: Show me all actions by a specific user — filter on materialized principal.id."""
_insert_standard_audit_events(insert_audit_logs)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = _query_audit_raw(
signoz,
token,
_build_audit_query(filter_expression="signoz.audit.principal.id = 'user-001'"),
)
assert response.status_code == HTTPStatus.OK
rows = response.json()["data"]["data"]["results"][0]["rows"]
assert len(rows) == 3 # alice: create, update, login
emails = [
row["data"]["attributes_string"]["signoz.audit.principal.email"] for row in rows
]
assert all(e == "alice@acme.com" for e in emails)
def test_audit_filter_by_outcome_failure(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_audit_logs: Callable[[List[AuditLog]], None],
) -> None:
"""Q2: Show me all failed actions — filter on materialized outcome."""
_insert_standard_audit_events(insert_audit_logs)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = _query_audit_raw(
signoz,
token,
_build_audit_query(filter_expression="signoz.audit.outcome = 'failure'"),
)
assert response.status_code == HTTPStatus.OK
rows = response.json()["data"]["data"]["results"][0]["rows"]
assert len(rows) == 1
row = rows[0]["data"]
assert row["attributes_string"]["signoz.audit.principal.email"] == "viewer@acme.com"
assert row["attributes_string"]["signoz.audit.action"] == "delete"
assert row["severity_text"] == "ERROR"
def test_audit_filter_by_resource(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_audit_logs: Callable[[List[AuditLog]], None],
) -> None:
"""Q3: Show me the change history of a specific dashboard — filter on resource.name + resource.id."""
_insert_standard_audit_events(insert_audit_logs)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = _query_audit_raw(
signoz,
token,
_build_audit_query(
filter_expression="signoz.audit.resource.name = 'dashboard' AND signoz.audit.resource.id = 'dash-001'"
),
)
assert response.status_code == HTTPStatus.OK
rows = response.json()["data"]["data"]["results"][0]["rows"]
assert len(rows) == 3 # create, update, failed delete — all on dash-001
actions = [row["data"]["attributes_string"]["signoz.audit.action"] for row in rows]
# Most recent first
assert actions == ["delete", "update", "create"]
def test_audit_filter_by_principal_type(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_audit_logs: Callable[[List[AuditLog]], None],
) -> None:
"""Q5: Show me all actions by service accounts — filter on materialized principal.type."""
_insert_standard_audit_events(insert_audit_logs)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = _query_audit_raw(
signoz,
token,
_build_audit_query(
filter_expression="signoz.audit.principal.type = 'service_account'"
),
)
assert response.status_code == HTTPStatus.OK
rows = response.json()["data"]["data"]["results"][0]["rows"]
assert len(rows) == 1
assert rows[0]["data"]["event_name"] == "serviceaccount.apikey.created"
def test_audit_scalar_count_failures(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_audit_logs: Callable[[List[AuditLog]], None],
) -> None:
"""Q6: Alert query — count failed actions (scalar aggregation)."""
_insert_standard_audit_events(insert_audit_logs)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
now = datetime.now(tz=timezone.utc)
response = make_query_request(
signoz,
token,
start_ms=int((now - timedelta(seconds=30)).timestamp() * 1000),
end_ms=int(now.timestamp() * 1000),
queries=[
_build_audit_ts_query(
aggregation="count()",
filter_expression="signoz.audit.outcome = 'failure'",
)
],
request_type="scalar",
)
assert response.status_code == HTTPStatus.OK
assert response.json()["status"] == "success"
results = response.json()["data"]["data"]["results"]
assert len(results) == 1
rows = results[0].get("rows", [])
assert len(rows) == 1
assert rows[0]["data"]["A"] == 1
def test_audit_does_not_leak_into_logs(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_audit_logs: Callable[[List[AuditLog]], None],
) -> None:
"""Audit data in signoz_audit must not appear when querying signal=logs without source=audit."""
_insert_standard_audit_events(insert_audit_logs)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
# Query regular logs (no source=audit) — should NOT see audit events
response = _query_audit_raw(
signoz,
token,
_build_audit_query(source=""),
)
assert response.status_code == HTTPStatus.OK
rows = response.json()["data"]["data"]["results"][0].get("rows", [])
# None of the audit events should appear in regular log queries
audit_bodies = [
row["data"]["body"]
for row in rows
if "signoz.audit"
in row["data"].get("attributes_string", {}).get("signoz.audit.action", "")
]
assert len(audit_bodies) == 0

View File

@@ -12,10 +12,10 @@ from fixtures.querier import (
assert_minutely_bucket_values,
build_builder_query,
find_named_result,
get_all_warnings,
get_error_message,
index_series_by_label,
make_query_request,
get_all_warnings,
get_error_message,
)
from fixtures.utils import get_testdata_file_path