Commit Graph

23 Commits

Author SHA1 Message Date
Pandey
42415e0873 feat(audit): handler-level AuditDef, audit middleware, and response capture (#10791)
* feat(audit): handler-level AuditDef and response-capturing wrapper

Add declarative audit instrumentation to the handler package. Routes
declare an AuditDef alongside OpenAPIDef; the handler automatically
captures the response status/body and emits an audit event via
auditor.Audit() after every request.

* refactor(audit): move audit logic to middleware, merge with logging

Move audit event emission from handler to middleware layer. The handler
package keeps only the AuditDef struct and AuditDefProvider interface.
The logging middleware now handles both request logging and audit event
emission using a single response capture, avoiding double-wrapping.

Rename badResponseLoggingWriter to responseCapture with body capture
on all 4xx/5xx responses (previously only 400 and 5xx).

* refactor(audit): rename Logging middleware to Audit, merge into single file

Delete logging.go and merge its contents into audit.go. Rename
Logging/NewLogging to Audit/NewAudit. The response.go file with
responseCapture is unchanged.

* refactor(audit): extract NewAuditEventFromHTTPRequest factory into audittypes

Move event construction to audittypes.NewAuditEventFromHTTPRequest with
an AuditEventContext struct for caller-provided fields. The audittypes
layer reads only transport fields from *http.Request and has no mux,
authtypes, or context dependencies. The middleware pre-extracts
principal, trace, error, and route fields before calling the factory.

* refactor(audit): move error parsing to render.ErrorFromBody and render.ErrorTypeFromStatusCode

Add render.ErrorFromBody to extract errors.JSON from a JSON-encoded
ErrorResponse body, and render.ErrorTypeFromStatusCode to reverse-map
HTTP status codes to error type strings. The middleware now uses these
instead of local duplicates.

* refactor(audit): move AuditDef onto Handler interface, consolidate files

Move AuditDef() onto the Handler interface directly. All Handler
implementations now carry it: handler returns the configured def,
healthOpenAPIHandler returns nil. Delete the separate AuditDefProvider
interface and audit.go handler file. Move excludedRoutes check before
audit emission so excluded routes skip both logging and audit.

* feat(audit): add option.go with AuditDef, Option, and WithAuditDef

* refactor(audit): decompose AuditEvent into attribute sub-structs, add tests

Decompose flat AuditEvent fields into typed sub-structs
(AuditEventAuditAttributes, PrincipalAttributes, ResourceAttributes,
ErrorAttributes, TransportAttributes) each with a constructor and
Put(pcommon.Map) method. Simplify NewAuditEventFromHTTPRequest to
accept authtypes.Claims and oteltrace IDs directly. Simplify the
middleware caller accordingly.

Add unit tests for the factory, outcome boundary, and principal type
derivation.

* refactor(audit): shorten attribute struct names, drop error message

Rename AuditEventAuditAttributes to AuditAttributes,
AuditEventPrincipalAttributes to PrincipalAttributes, and likewise
for Resource, Error, and Transport. The package prefix already
disambiguates.

Remove ErrorMessage from ErrorAttributes to avoid leaking sensitive
or PII data into audit logs. Error type and code are sufficient for
filtering; investigators can correlate via trace ID.

* fix(audit): update auditorserver test and otlphttp provider for new struct layout

Update newTestEvent in server_test.go to use nested AuditAttributes
and ResourceAttributes. Update otlphttpauditor provider to access
PrincipalOrgID via PrincipalAttributes. Fix godot lint on attribute
section comments.

* fix(audit): fix gjson path in ErrorCodeFromBody, add tests

Fix ErrorCodeFromBody gjson path from "errors.code" to "error.code"
to match the ErrorResponse JSON structure. Add unit tests for valid
error response and invalid JSON cases.

* fix(audit): add CodeUnset, use ErrorCodeFromBody in middleware

Add errors.CodeUnset for responses missing an error code. Update the
audit middleware to use render.ErrorCodeFromBody instead of the removed
render.ErrorFromBody.

* test(audit): add unit tests for responseCapture

Test the four meaningful behaviors: success responses don't capture
body, error responses capture body, large error bodies truncate at
4096 bytes, and 204 No Content suppresses writes entirely.

* fix(audit): check rw.Write return values in response_test.go

* style(audit): rename want prefix to expected in test fields

* refactor(audit): replace Sprintf with strings.Builder in newBody

Handle edge cases where principal email, ID, or resource ID may be
empty. The builder conditionally includes each segment, avoiding
empty parentheses or leading spaces in the audit body.

Add test cases covering all meaningful combinations: success/failure
with full/partial/empty principal, resource ID, and error details.

* chore: fix formatting

* chore: remove json tags

* fix: rebase with main
2026-04-01 10:10:52 +00:00
Vikrant Gupta
2163e1ce41 chore(lint): enable godot and staticcheck (#10775)
* chore(lint): enable godot and staticcheck

* chore(lint): merge main and fix new lint issues in main
2026-03-31 09:11:49 +00:00
Pandey
b811991f9d feat(middleware): add panic recovery middleware (#10666)
* feat(middleware): add panic recovery middleware with TypeFatal error type

Add a global HTTP recovery middleware that catches panics, logs them
with OTel exception semantic conventions via errors.Attr, and returns
a safe user-facing error response. Introduce TypeFatal/CodeFatal for
unrecoverable failures and WithStacktrace to attach pre-formatted
stack traces to errors. Remove redundant per-handler panic recovery
blocks in querier APIs.

* style(errors): keep WithStacktrace call on same line in test

* fix(middleware): replace fmt.Errorf with errors.New in recovery test

* feat(middleware): add request context to panic recovery logs

Capture request body before handler runs and include method, path, and
body in panic recovery logs using OTel semconv attributes. Improve error
message to direct users to GitHub issues or support.
2026-03-23 06:25:26 +00:00
Pandey
95ed125bd9 feat(instrumentation): add OTel exception semantic convention log handler (#10665)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
* feat(instrumentation): add OTel exception semantic convention log handler

Add a loghandler.Wrapper that enriches error log records with OpenTelemetry
exception semantic convention attributes (exception.type, exception.code,
exception.message, exception.stacktrace).

- Add errors.Attr() helper for standardized error logging under "exception" key
- Add exception log handler that replaces raw error attrs with structured group
- Wire exception handler into the instrumentation SDK logger chain
- Remove LogValue() from errors.base as the handler now owns structuring

* refactor: replace "error", err with errors.Attr(err) across codebase

Migrate all slog error logging from ad-hoc "error", err key-value pairs
to the standardized errors.Attr(err) helper, enabling the exception log
handler to enrich these logs with OTel semantic convention attributes.

* refactor: enforce attr-only slog style across codebase

Change sloglint from kv-only to attr-only, requiring all slog calls to
use typed attributes (slog.String, slog.Any, etc.) instead of key-value
pairs. Convert all existing kv-style slog calls in non-excluded paths.

* refactor: tighten slog.Any to specific types and standardize error attrs

- Replace slog.Any with slog.String for string values (action, key, where_clause)
- Replace slog.Any with slog.Uint64 for uint64 values (start, end, step, etc.)
- Replace slog.Any("err", err) with errors.Attr(err) in dispatcher and segment analytics
- Replace slog.Any("error", ctx.Err()) with errors.Attr in factory registry

* fix(instrumentation): use Unwrapb message for exception.message

Use the explicit error message (m) from Unwrapb instead of
foundErr.Error(), which resolves to the inner cause's message
for wrapped errors.

* feat(errors): capture stacktrace at error creation time

Store program counters ([]uintptr) in base errors at creation time
using runtime.Callers, inspired by thanos-io/thanos/pkg/errors. The
exception log handler reads the stacktrace from the error instead of
capturing at log time, showing where the error originated.

* fix(instrumentation): apply default log wrappers uniformly in NewLogger

Move correlation, filtering, and exception wrappers into NewLogger so
all call sites (including CLI loggers in cmd/) get them automatically.

* refactor(instrumentation): remove variadic wrappers from NewLogger

NewLogger no longer accepts arbitrary wrappers. The core wrappers
(correlation, filtering, exception) are hardcoded, preventing callers
from accidentally duplicating behavior.

* refactor: migrate remaining "error", <var> to errors.Attr across legacy paths

Replace all remaining "error", <variable> key-value pairs with
errors.Attr(<variable>) in pkg/query-service/ and ee/query-service/
paths that were missed in the initial migration due to non-standard
variable names (res.Err, filterErr, apiErrorObj.Err, etc).

* refactor(instrumentation): use flat exception.* keys instead of nested group

Use flat keys (exception.type, exception.code, exception.message,
exception.stacktrace) instead of a nested slog.Group in the exception
log handler.
2026-03-22 04:06:31 +00:00
Vikrant Gupta
465e07de83 fix(openapi): make the error and status as mandatory (#10391)
* fix(openapi): make the error and status as mandatory

* fix(openapi): fix the frontend types
2026-02-23 16:47:24 +05:30
Karan Balani
6d137bcdff feat: idp attributes mapping (#9841) 2026-01-19 22:27:21 +05:30
Piyush Singariya
bca761498a chore(JSON): Promote Body Paths API (#9592) 2025-12-23 14:11:52 +05:30
Piyush Singariya
e66bfe5961 feat(JSON): JSON Body Metadata (#9593)
* feat: json Body Keys

* feat: telemetry types

* feat: change ExtractBodyPaths

* chore: minor comment change

* chore: func rename, file rename

* chore: change table names

* chore: reflect changes from the overhaul

* test: fixing test 1

* fix: test TestQueryToKeys

* fix: test TestPrepareLogsQuery

* chore: remove db

* chore: go mod

* chore: changes based on review

* chore: changes based on review

* fix: in LIKE operation

* chore: addressed few changes

* revert: test file

* fix: comparison fix

* test: add TestBuildListLogsJSONIndexesQuery

* fix: in test TestBuildListLogsJSONIndexesQuery

* fix: pull promoted paths in single db call

* fix: reducing db calls

* test: fix TestBuildListLogsJSONIndexesQuery

* fix: test TestConditionForJSONBodySearch

* fix: lint try 1

* chore: review changes based on cursor

* fix: use enums only

---------

Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
Co-authored-by: Nityananda Gohain <nityanandagohain@gmail.com>
2025-12-09 20:47:26 +07:00
Pranjul Kalsi
bdce97a727 fix: replace fmt.Errorf with signoz/pkg/errors and update golangci-li… (#9373)
This PR fulfills the requirements of #9069 by:

- Adding a golangci-lint directive (forbidigo) to disallow all fmt.Errorf usages.
- Replacing existing fmt.Errorf instances with structured errors from github.com/SigNoz/signoz/pkg/errors for consistent error classification and lint compliance.
- Verified lint and build integrity.
2025-10-27 16:30:18 +05:30
Vibhu Pandey
c122bc09b4 feat(tokenizer|sso): add tokenizer for session management and oidc sso support (#9183)
## 📄 Summary

- Instead of relying on JWT for session management, we are adding another token system: opaque. This gives the benefits of expiration and revocation.

- We are now ensuring that emails are regex checked throughout the backend.

- Support has been added for OIDC protocol
2025-10-16 18:00:38 +05:30
Vibhu Pandey
c83eaf3d50 chore: enable forbidigo and noerrors in depguard (#9047)
* chore: enable forbidgo

* chore: enable forbidgo
2025-09-09 15:44:27 +05:30
Vikrant Gupta
f61e859901 feat(authz): embed openfga server (#8966)
* feat(access-control): embed openfga in signoz

* feat(authz): rename access control to authz

* feat(authz): fix codeowners and go mod tidy

* feat(authz): fix lint

* feat(authz): update go version and move convertor to instrumentation

* feat(authz): some more lint issues

* feat(authz): some more lint issues

* feat(authz): some more lint issues

* feat(authz): fix more lint issues

* feat(authz): make logger converter interface
2025-09-01 17:10:13 +05:30
Piyush Singariya
d6eed8e79d feat: JSON Flattening in logs pipelines (#8227)
* feat: introducing JSON Flattening

* fix: removed bug and tested

* test: removed testing test

* feat: additional severity levels, and some clearing

* chore: minor changes

* test: added tests for processJSONParser

* test: added check for OnError

* fix: review from ellipsis

* fix: variablise max flattening depth

* Update pkg/query-service/app/logparsingpipeline/pipelineBuilder.go

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update pkg/errors/errors.go

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* fix: quoted JSON strings fix

* test: updating otel collector for testing

* test: update collector's reference

* chore: change with new error package

* chore: set flattening depth equal to 1

* fix: fallback for depth

* fix: change in errors package

* fix: tests

* fix: test

* chore: update collector version

* fix: go.sum

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: Nityananda Gohain <nityanandagohain@gmail.com>
2025-07-14 18:48:01 +05:30
Srikanth Chekuri
72b0214d1d chore: add range query impl for promql (#8130) 2025-06-05 19:18:44 +00:00
Srikanth Chekuri
c5d5c84a0e chore: add fieldmapper implementation (#7955) 2025-05-16 20:09:57 +05:30
Piyush Singariya
03ab6e704b feat: S3 Sync (AWS Integrations) (#7718) 2025-05-14 05:12:41 +05:30
Vibhu Pandey
5bceffbeaa fix: fix modules and handler (#7737)
* fix: fix modules and handler

* fix: fix sqlmigration package

* fix: fix other fmt issues

* fix: fix tests

* fix: fix tests
2025-04-27 16:38:34 +05:30
Nityananda Gohain
a9618886b9 fix: support multitenancy in dashboards & savedviews (#7237)
* fix: support multitenancy in dashboards

* fix: support multitenancy in saved views

* fix: move migrations to provider file

* fix: remove getUserFromClaims and use new errors

* fix: remove getUserFromClaims and use new errors

* fix: use new errors in dashboards.go

* Update ee/query-service/app/api/dashboard.go

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* fix: minor changes

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-03-12 17:18:11 +05:30
Vibhu Pandey
918c8942c4 feat(alertmanager): add service for alertmanager (#7136)
### Summary

- adds an alertmanager service
2025-02-18 07:36:31 +00:00
Vibhu Pandey
bcf7bf38fc feat(alertmanager): add alertmanagertypes (#7101)
add alertmanagertypes
2025-02-12 17:23:18 +00:00
Vibhu Pandey
001122db2c feat(instrumentation): adopt slog (#6907)
### Summary

feat(instrumentation): adopt slog
2025-01-24 09:23:02 +00:00
Vibhu Pandey
bd7d14b1ca feat(render): add render package (#5751)
### Summary

Add `render` package

#### Related Issues / PR's

https://github.com/SigNoz/signoz/pull/5710
2024-08-23 13:07:10 +05:30
Vibhu Pandey
c322fc72d9 feat(errors): add errors package (#5741)
### Summary

Add errors package

#### Related Issues / PR's

https://github.com/SigNoz/signoz/pull/5710
2024-08-22 15:19:32 +05:30