Compare commits

..

33 Commits

Author SHA1 Message Date
Nikhil Soni
975683e075 fix(query-service): fail startup when opamp server cannot start
opampServer.Start returns the listen error synchronously, so run it inline
and propagate. The error was previously swallowed in a goroutine; the old
unavailableChannel it replaced had no reader and only leaked a blocked
goroutine, so a bind failure never brought the process down.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-09 14:45:30 +05:30
Nikhil Soni
ebf4a0f4ee feat(apiserver): split http server read/write timeout
The single connection timeout drove both read and write. A write timeout
severs streaming endpoints (livetail, SSE, export_raw_data), so the server
now takes separate read/write timeouts and the apiserver sets only the read
side via apiserver.timeout.request_read.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-09 14:45:29 +05:30
Nikhil Soni
c16d93afef fix(apiserver): validate address in config.Validate
Openapi spec generation builds the apiserver with an empty config, so the
address requirement belongs at config validation, not provider construction.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-09 14:04:29 +05:30
Nikhil Soni
c36722a26f fix(http): drop dead websocket cors headers
The relocated cors middleware still allowed X-SIGNOZ-QUERY-ID and
Sec-WebSocket-Protocol, both removed with query progress tracking in #12784.

Assisted-by: Claude Opus 4.8
2026-09-08 18:03:32 +05:30
Nikhil Soni
d5854902d3 Merge remote-tracking branch 'origin/main' into ns/harness
# Conflicts:
#	ee/query-service/app/server.go
#	pkg/query-service/app/server.go
2026-09-08 17:25:09 +05:30
Nikhil Soni
98821e7f06 fix(signoz): build openapi apiserver with the default config
Moving the apiserver.address validation into newProvider broke openapi
spec generation, which constructs the apiserver with an empty config
only to walk its routes and never starts the server. Use the default
config from the config factory instead.

Assisted-by: kimi-k3
2026-09-07 19:26:36 +05:30
Nikhil Soni
8592b8afea refactor(apiserver): reuse http/server for the apiserver lifecycle
http/server timeout is configurable; apiserver opts out (0) to keep
long-lived routes (websockets, livetail, export) unbounded, pprof keeps 10s.

Assisted-by: Claude Opus 4.8
2026-09-07 19:00:30 +05:30
Nikhil Soni
4b97648b74 refactor(http): extract cors, compress and otel middlewares
Otel middleware excludes the health endpoints from tracing by default.

Assisted-by: Claude Opus 4.8
2026-09-07 18:54:55 +05:30
Nikhil Soni
a5817a890f chore(query-service): remove unused healthcheck package
Assisted-by: Claude Opus 4.8
2026-09-07 18:53:41 +05:30
Nikhil Soni
e567d33d54 Merge remote-tracking branch 'origin/main' into ns/harness
# Conflicts:
#	pkg/apiserver/signozapiserver/provider.go
#	pkg/signoz/openapi.go
#	pkg/signoz/provider.go
#	pkg/signoz/provider_test.go
#	pkg/signoz/signoz.go
2026-09-07 17:35:44 +05:30
Nikhil Soni
e99263d52e refactor(apiserver): own the http server lifecycle in the signozapiserver provider
The apiserver was constructed via the provider factory but its http
server was started by the query-service outside the factory registry,
so it never showed up in /api/v2/healthz.

- signozapiserver provider now implements factory.ServiceWithHealthy:
  it builds the middleware chain, cors, compression and external-path
  handling, binds the listener in Start (closing healthyC once bound)
  and shuts down gracefully in Stop
- apiserver.APIServer embeds factory.ServiceWithHealthy
- registry gains Add() so the apiserver, which depends on the
  registry-backed health handler, can be registered after construction
- query-service (community and ee) registers its legacy routes on the
  apiserver router and keeps only the opamp websocket server
2026-09-02 18:01:12 +05:30
Nikhil Soni
aa46493d8a chore: add opencode config to use with OPENCODE_CONFIG in sandbox 2026-09-02 15:35:30 +05:30
Nikhil Soni
7fb1feec9d chore: pick a rule from HN post for commenting
Link: https://fabiensanglard.net/agent.md/index.html
2026-08-24 11:33:06 +05:30
Nikhil Soni
60e949d8d8 chore: add rule for avoid commit amend 2026-08-19 22:34:30 +05:30
Nikhil Soni
474aabfd36 chore: make commands for running on worktree discoverable 2026-08-18 15:31:10 +05:30
Nikhil Soni
887b8afd60 docs(contributing): document running multiple signoz instances 2026-08-18 15:21:00 +05:30
Nikhil Soni
37364f611b Revert "chore(hooks): replace husky with worktree-safe .githooks installed automatically"
This reverts commit 1accf43a95.
2026-08-18 15:13:44 +05:30
Nikhil Soni
2b4e1ba382 chore: add rule for running gofmt 2026-08-18 15:13:44 +05:30
Nikhil Soni
9ba63c0a06 chore(hooks): replace husky with worktree-safe .githooks installed automatically
Husky wired hook installation to pnpm install, which go-only contributors
never run. The installer is plain shell so both the Makefile and the frontend
postinstall can call it, and each check is gated on staged paths so neither
side needs the other's toolchain.
2026-08-18 15:13:44 +05:30
Nikhil Soni
206c6f4627 revert(hooks): drop root .husky move and the staged-go gofmt step
Hook installation strategy still needs deciding; reverting to keep it out of
this PR.
2026-08-18 15:13:44 +05:30
Nikhil Soni
342465e715 chore(makefile): poll go-stop once a second so the timeout reads as one number 2026-08-18 15:13:44 +05:30
Nikhil Soni
3bf6aca021 fix(makefile): make go-stop wait for the server to actually exit
kill only signals; the process keeps every port bound while it shuts down
gracefully, so an immediate restart failed on the prometheus exporter port.
2026-08-18 15:13:44 +05:30
Nikhil Soni
0680edd23f chore(makefile): rely on env inheritance for signoz overrides in go-run targets 2026-08-18 15:13:44 +05:30
Nikhil Soni
fd71c555ed chore(hooks): move git hooks from frontend/.husky to root .husky 2026-08-18 15:13:44 +05:30
Nikhil Soni
8bb9fa1a49 chore(makefile): hint at overriding SIGNOZ_APISERVER_ADDRESS when go-stop finds nothing 2026-08-18 15:13:44 +05:30
Nikhil Soni
572fc3e2e2 chore(makefile): rename HTTP_HOST_PORT to SIGNOZ_APISERVER_ADDRESS 2026-08-18 15:13:44 +05:30
Nikhil Soni
bcb315009d chore(hooks): gofmt staged Go files on commit 2026-08-18 15:13:44 +05:30
Nikhil Soni
371a8459ff revert(config): keep opamp websocket port hardcoded, only apiserver port is configurable 2026-08-18 15:13:44 +05:30
Nikhil Soni
7f77681a2b refactor(config): move signoz port config from raw env vars to structured config 2026-08-18 15:13:44 +05:30
Nikhil Soni
70f192f779 chore: add opinionated contrib rule 2026-08-18 15:13:44 +05:30
Nikhil Soni
45158fa046 chore: include commit message instruction as well 2026-08-18 15:13:44 +05:30
Nikhil Soni
78eee04b18 chore: add local claude settings in ignore 2026-08-18 15:13:43 +05:30
Nikhil Soni
84a9b9f346 chore(dev): make signoz http/private/opamp ports and sqlite path configurable 2026-08-18 15:13:43 +05:30
70 changed files with 820 additions and 2569 deletions

7
.claude/opencode.json Normal file
View File

@@ -0,0 +1,7 @@
{
"$schema": "https://opencode.ai/config.json",
"lsp": true,
"experimental": {
"disable_paste_summary": true
}
}

View File

@@ -7,5 +7,6 @@ Applies to everything in the repo — code, config, workflows.
- **Rationale goes in prose, not source.** Why a version is pinned, why a job exists, how a subsystem fits together — that belongs in the README or the PR.
- **Never remove pre-existing comments** when editing code. The bar above applies to comments you write, not comments already there.
- **Never talk to the reviewer.** No comments about where a change came from, what was changed, or why the change is correct — that belongs in the PR description and is noise the moment it merges.
- **Less is more.** When writing something intended for human consumption, (comment, commit message, reply to prompt) use as few words as possible. Pick every word meticulously to reduce the volume to a strict minimum. Be down to the point. Less is more.
Language rules build on this one: [`go-comments`](go-comments.md), [`py-comments`](py-comments.md).

View File

@@ -0,0 +1,16 @@
---
paths:
- "**/*.go"
---
# Contribution guidelines
- When making Go changes, always ensure they follow the contributing guildelines in [`docs/contributing/go/`](../../docs/contributing/go/).
- Look for existing patterns in the codebase for any change before implementing the changes.
- If any API contract is modified, generate the OpenAPI specs with `make gen-openapi-specs`.
- Always keep the OpenAPI spec generated in a separate commit, so the whole commit can be dropped in case of conflicts during merge. Do not try to resolve conflict in generated files, instead just generate them again.
- Avoid breaking function calls unncessarily into multilines for couple of arguments.
- Try to keep most computational only logic in types package itself related to a domain type, use modules as the orchestraction layer cordinating different layers and all db queries in store layer. Check the serviceaccount modules for inspiration when confused.
- When defining types, keep the structure of file to have any constants and variables first, then exported types and exported methods and then finally the unexported types and methods.
- Never import types or other modules in migration files, duplicate the required type or method to keep migration free from changes.
- Always run the gofmt tool for formating beforing commiting any changes.

View File

@@ -2,6 +2,10 @@
- **Follow the template** (`.github/pull_request_template.md`): fill in its headings (Description / Issues closed by this PR / Screenshots / Additional Information). Don't add sections the template doesn't have.
- **Keep only the headings that apply.** Delete every heading that has nothing under it, along with its `<!--...-->` placeholder comment. The body must never contain an empty heading — if only Description applies, the body has exactly that one heading.
- **Keep the description concise and human-readable.** A few plain bullets saying what changed and why, for a reviewer skimming it — not a wall of text, not a restatement of the diff, not generated boilerplate.
- **Keep the description concise and human-readable.** A few non repetitive bullets saying what changed and why, for a reviewer skimming it — not a wall of text, not a restatement of the diff, not generated boilerplate and not the user agent conversation details.
- **Reference issues with `Closes #issue-number`** under "Issues closed by this PR" so they auto-close on merge. This goes in the PR description only — never in commit messages.
- **Breaking changes can be added in additional information section** if any.
- **AI assistance in commits may optionally be disclosed with an `Assisted-by:` trailer** naming the model (e.g. `Assisted-by: Claude Opus 4.5`) — do NOT use a `Co-authored-by:` trailer for this.
- **Keep the commit body short and human readable** focused on decision made if any. Commit body must not re-iterate the changes done, skip if title is sufficient in conveying the change.
- **Use convensional commit format** for commits and PR title.
- **Do not amend the commits once pushed.** Always create a new commit once changes are pushed to remote.

1
.gitignore vendored
View File

@@ -232,3 +232,4 @@ pyrightconfig.json
# dev
.dev/
.claude/worktrees/
.claude/settings.local.json

View File

@@ -81,10 +81,13 @@ devenv-clickhouse-clean: ## Clean all ClickHouse data from filesystem
##############################################################
# go commands
##############################################################
SIGNOZ_SQLSTORE_SQLITE_PATH ?= signoz.db
SIGNOZ_APISERVER_ADDRESS ?= 0.0.0.0:8080
.PHONY: go-run-enterprise
go-run-enterprise: ## Runs the enterprise go backend server
@SIGNOZ_INSTRUMENTATION_LOGS_LEVEL=debug \
SIGNOZ_SQLSTORE_SQLITE_PATH=signoz.db \
SIGNOZ_SQLSTORE_SQLITE_PATH=$(SIGNOZ_SQLSTORE_SQLITE_PATH) \
SIGNOZ_WEB_ENABLED=false \
SIGNOZ_TOKENIZER_JWT_SECRET=secret \
SIGNOZ_ALERTMANAGER_PROVIDER=signoz \
@@ -101,7 +104,7 @@ go-test: ## Runs go unit tests
.PHONY: go-run-community
go-run-community: ## Runs the community go backend server
@SIGNOZ_INSTRUMENTATION_LOGS_LEVEL=debug \
SIGNOZ_SQLSTORE_SQLITE_PATH=signoz.db \
SIGNOZ_SQLSTORE_SQLITE_PATH=$(SIGNOZ_SQLSTORE_SQLITE_PATH) \
SIGNOZ_WEB_ENABLED=false \
SIGNOZ_TOKENIZER_JWT_SECRET=secret \
SIGNOZ_ALERTMANAGER_PROVIDER=signoz \
@@ -111,6 +114,28 @@ go-run-community: ## Runs the community go backend server
go run -race \
$(GO_BUILD_CONTEXT_COMMUNITY)/*.go server
.PHONY: go-stop
go-stop: ## Stops the go backend server listening on SIGNOZ_APISERVER_ADDRESS, waiting for it to release every port it holds
@PORT=$(lastword $(subst :, ,$(SIGNOZ_APISERVER_ADDRESS))); \
PIDS=$$(lsof -ti tcp:$$PORT); \
if [ -z "$$PIDS" ]; then \
echo "No signoz server running on port $$PORT."; \
echo "If it's running on a different port, rerun as: make go-stop SIGNOZ_APISERVER_ADDRESS=host:port"; \
exit 0; \
fi; \
kill $$PIDS 2>/dev/null; \
for i in $$(seq 1 10); do \
alive=$$(for p in $$PIDS; do kill -0 $$p 2>/dev/null && echo $$p; done); \
[ -z "$$alive" ] && break; \
sleep 1; \
done; \
alive=$$(for p in $$PIDS; do kill -0 $$p 2>/dev/null && echo $$p; done); \
if [ -n "$$alive" ]; then \
echo "Graceful shutdown did not finish in 10s, sending SIGKILL to $$alive"; \
kill -9 $$alive 2>/dev/null; \
fi; \
echo "Stopped signoz server on port $$PORT (pid $$PIDS)"
.PHONY: go-build-community $(GO_BUILD_ARCHS_COMMUNITY)
go-build-community: ## Builds the go backend server for community
go-build-community: $(GO_BUILD_ARCHS_COMMUNITY)
@@ -241,3 +266,8 @@ semconv-generate: ## Regenerate semantic-convention families for Go and TypeScri
gen-mocks:
@echo ">> Generating mocks"
@mockery --config .mockery.yml
.PHONY: gen-openapi-specs
gen-openapi-specs:
@go run cmd/enterprise/*.go generate openapi
cd frontend && pnpm generate:api && cd -

View File

@@ -102,7 +102,6 @@ func runGenerateAuthz(_ context.Context) error {
coretypes.NewResourceRef(coretypes.ResourceTelemetryResourceTraces).String(): true,
coretypes.NewResourceRef(coretypes.ResourceTelemetryResourceMetrics).String(): true,
coretypes.NewResourceRef(coretypes.ResourceTelemetryResourceMeterMetrics).String(): true,
coretypes.NewResourceRef(coretypes.ResourceMetaResourceQuickFilter).String(): true,
}
allowedTypes := map[string]bool{}

View File

@@ -138,11 +138,15 @@ sqlstore:
##################### APIServer #####################
apiserver:
# The TCP address the API server listens on, in the form "host:port".
address: 0.0.0.0:8080
timeout:
# Default request timeout.
default: 60s
# Maximum request timeout.
max: 600s
# Maximum duration for reading an entire request, including the body.
request_read: 60s
# List of routes to exclude from request timeout.
excluded_routes:
- /api/v1/logs/tail

View File

@@ -83,7 +83,13 @@ This command:
You should see: `{"status":"ok"}`
> 💡 **Tip**: The API server runs at `http://localhost:8080/` by default
3. Stop it when you're done:
```bash
make go-stop
```
> 💡 **Tip**: The API server runs at `http://localhost:8080/` by default. You can configure this using `apiserver.address` configuration option. See
> [running more than one instance](#how-do-i-run-more-than-one-instance) if you need that for agentic testing.
### 4. Setting up the Frontend
@@ -119,6 +125,36 @@ To verify everything is working correctly:
3. **Check Backend**: `curl http://localhost:8080/api/v1/health` (should return `{"status":"ok"}`)
4. **Check Frontend**: Open `http://localhost:3301` in your browser
## How do I run more than one instance?
Handy when you keep several branches checked out as separate git worktrees. Every port
and path below is read from the environment, so set them on the `make` call:
```bash
SIGNOZ_APISERVER_ADDRESS=0.0.0.0:8081 \
SIGNOZ_SQLSTORE_SQLITE_PATH=/path/to/main/sqlite.db \
SIGNOZ_INSTRUMENTATION_METRICS_READERS_PULL_EXPORTER_PROMETHEUS_PORT=9091 \
make go-run-community
```
| Variable | Default | Why you'd change it |
| --- | --- | --- |
| `SIGNOZ_APISERVER_ADDRESS` | `0.0.0.0:8080` | Address the API server listens on |
| `SIGNOZ_SQLSTORE_SQLITE_PATH` | `signoz.db` in worktree | To reuse same database |
| `SIGNOZ_INSTRUMENTATION_METRICS_READERS_PULL_EXPORTER_PROMETHEUS_PORT` | `9090` | Bound by the Prometheus metrics exporter on startup |
Point the frontend at whichever backend you want, in `frontend/.env`:
```env
VITE_FRONTEND_API_ENDPOINT=http://localhost:8081
```
Stop an instance using the address it was started on:
```bash
make go-stop SIGNOZ_APISERVER_ADDRESS=0.0.0.0:8081
```
## How to send test data?
You can now send telemetry data to your local SigNoz instance:

View File

@@ -3,56 +3,29 @@ package app
import (
"context"
"fmt"
"net"
"net/http"
"slices"
"go.opentelemetry.io/contrib/instrumentation/github.com/gorilla/mux/otelmux"
"go.opentelemetry.io/otel/propagation"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/gorilla/handlers"
"github.com/rs/cors"
"github.com/soheilhy/cmux"
"github.com/SigNoz/signoz/ee/query-service/app/api"
"github.com/SigNoz/signoz/ee/query-service/usage"
"github.com/SigNoz/signoz/pkg/http/middleware"
"github.com/SigNoz/signoz/pkg/signoz"
"github.com/SigNoz/signoz/pkg/web"
"log/slog"
"github.com/SigNoz/signoz/pkg/query-service/agentConf"
baseapp "github.com/SigNoz/signoz/pkg/query-service/app"
"github.com/SigNoz/signoz/pkg/query-service/app/clickhouseReader"
"github.com/SigNoz/signoz/pkg/query-service/app/integrations"
"github.com/SigNoz/signoz/pkg/query-service/app/logparsingpipeline"
"github.com/SigNoz/signoz/pkg/query-service/app/opamp"
opAmpModel "github.com/SigNoz/signoz/pkg/query-service/app/opamp/model"
baseconst "github.com/SigNoz/signoz/pkg/query-service/constants"
"github.com/SigNoz/signoz/pkg/query-service/healthcheck"
"github.com/SigNoz/signoz/pkg/query-service/utils"
)
// Server runs HTTP, Mux and a grpc server
// Server runs auxiliary servers (opamp) alongside the signoz apiserver
type Server struct {
config signoz.Config
signoz *signoz.SigNoz
// public http router
httpConn net.Listener
httpServer *http.Server
httpHostPort string
opampServer *opamp.Server
// Usage manager
usageManager *usage.Manager
unavailableChannel chan healthcheck.Status
}
// NewServer creates and initializes Server
@@ -127,57 +100,11 @@ func NewServer(config signoz.Config, signoz *signoz.SigNoz) (*Server, error) {
return nil, err
}
s := &Server{
config: config,
signoz: signoz,
httpHostPort: baseconst.HTTPHostPort,
unavailableChannel: make(chan healthcheck.Status),
usageManager: usageManager,
}
httpServer, err := s.createPublicServer(apiHandler, signoz.Web)
if err != nil {
return nil, err
}
s.httpServer = httpServer
s.opampServer = opamp.InitializeServer(
&opAmpModel.AllAgents, agentConfMgr, signoz.Instrumentation,
)
return s, nil
}
// HealthCheckStatus returns health check status channel a client can subscribe to
func (s Server) HealthCheckStatus() chan healthcheck.Status {
return s.unavailableChannel
}
func (s *Server) createPublicServer(apiHandler *api.APIHandler, web web.Web) (*http.Server, error) {
r := baseapp.NewRouter()
am := middleware.NewAuthZ(s.signoz.Instrumentation.Logger(), s.signoz.Modules.OrgGetter, s.signoz.Authz)
r.Use(middleware.NewRecovery(s.signoz.Instrumentation.Logger()).Wrap)
r.Use(otelmux.Middleware(
"apiserver",
otelmux.WithMeterProvider(s.signoz.Instrumentation.MeterProvider()),
otelmux.WithTracerProvider(s.signoz.Instrumentation.TracerProvider()),
otelmux.WithPropagators(propagation.NewCompositeTextMapPropagator(propagation.Baggage{}, propagation.TraceContext{})),
otelmux.WithFilter(func(r *http.Request) bool {
return !slices.Contains([]string{"/api/v1/health"}, r.URL.Path)
}),
))
r.Use(middleware.NewIdentN(s.signoz.IdentNResolver, s.signoz.Sharder, s.signoz.Instrumentation.Logger()).Wrap)
r.Use(middleware.NewTimeout(s.signoz.Instrumentation.Logger(),
s.config.APIServer.Timeout.ExcludedRoutes,
s.config.APIServer.Timeout.Default,
s.config.APIServer.Timeout.Max,
).Wrap)
r.Use(middleware.NewResource(s.signoz.Instrumentation.Logger()).Wrap)
r.Use(middleware.NewAudit(s.signoz.Instrumentation.Logger(), s.config.APIServer.Logging.ExcludedRoutes, s.signoz.Auditor).Wrap)
r.Use(middleware.NewComment().Wrap)
// Register the legacy query-service routes on the apiserver router. The
// apiserver owns the HTTP server and applies the middleware chain at serve
// time, so these routes get the same treatment as the apiserver routes.
r := signoz.APIServer.Router()
am := middleware.NewAuthZ(signoz.Instrumentation.Logger(), signoz.Modules.OrgGetter, signoz.Authz)
apiHandler.RegisterRoutes(r, am)
apiHandler.RegisterLogsRoutes(r, am)
@@ -188,107 +115,29 @@ func (s *Server) createPublicServer(apiHandler *api.APIHandler, web web.Web) (*h
apiHandler.RegisterThirdPartyApiRoutes(r, am)
apiHandler.RegisterTraceFunnelsRoutes(r, am)
err := s.signoz.APIServer.AddToRouter(r)
if err != nil {
return nil, err
s := &Server{
usageManager: usageManager,
}
c := cors.New(cors.Options{
AllowedOrigins: []string{"*"},
AllowedMethods: []string{"GET", "DELETE", "POST", "PUT", "PATCH", "OPTIONS"},
AllowedHeaders: []string{"Accept", "Authorization", "Content-Type", "cache-control"},
})
s.opampServer = opamp.InitializeServer(
&opAmpModel.AllAgents, agentConfMgr, signoz.Instrumentation,
)
handler := c.Handler(r)
handler = handlers.CompressHandler(handler)
err = web.AddToRouter(r)
if err != nil {
return nil, err
}
routePrefix := s.config.Global.ExternalPath()
if routePrefix != "" {
prefixed := http.StripPrefix(routePrefix, handler)
handler = http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
switch req.URL.Path {
case "/api/v1/health", "/api/v2/healthz", "/api/v2/readyz", "/api/v2/livez":
r.ServeHTTP(w, req)
return
}
prefixed.ServeHTTP(w, req)
})
}
return &http.Server{
Handler: handler,
}, nil
return s, nil
}
// initListeners initialises listeners of the server
func (s *Server) initListeners() error {
// listen on public port
var err error
publicHostPort := s.httpHostPort
if publicHostPort == "" {
return fmt.Errorf("baseconst.HTTPHostPort is required")
}
s.httpConn, err = net.Listen("tcp", publicHostPort)
if err != nil {
return err
}
slog.Info(fmt.Sprintf("Query server started listening on %s...", s.httpHostPort))
return nil
}
// Start listening on http and private http port concurrently
// Start starts the opamp websocket server. The HTTP API server is started by
// the signoz registry.
func (s *Server) Start(ctx context.Context) error {
err := s.initListeners()
if err != nil {
slog.Info("Starting OpAmp Websocket server", "addr", baseconst.OpAmpWsEndpoint)
if err := s.opampServer.Start(baseconst.OpAmpWsEndpoint); err != nil {
return err
}
var httpPort int
if port, err := utils.GetPort(s.httpConn.Addr()); err == nil {
httpPort = port
}
go func() {
slog.Info("Starting HTTP server", "port", httpPort, "addr", s.httpHostPort)
switch err := s.httpServer.Serve(s.httpConn); err {
case nil, http.ErrServerClosed, cmux.ErrListenerClosed:
// normal exit, nothing to do
default:
slog.Error("Could not start HTTP server", errors.Attr(err))
}
s.unavailableChannel <- healthcheck.Unavailable
}()
go func() {
slog.Info("Starting OpAmp Websocket server", "addr", baseconst.OpAmpWsEndpoint)
err := s.opampServer.Start(baseconst.OpAmpWsEndpoint)
if err != nil {
slog.Error("opamp ws server failed to start", errors.Attr(err))
s.unavailableChannel <- healthcheck.Unavailable
}
}()
return nil
}
func (s *Server) Stop(ctx context.Context) error {
if s.httpServer != nil {
if err := s.httpServer.Shutdown(ctx); err != nil {
return err
}
}
s.opampServer.Stop()
// stop usage manager

View File

@@ -20,6 +20,7 @@ export const Logout = async (): Promise<void> => {
deleteLocalStorageKey(LOCALSTORAGE.LOGGED_IN_USER_NAME);
deleteLocalStorageKey(LOCALSTORAGE.CHAT_SUPPORT);
deleteLocalStorageKey(LOCALSTORAGE.USER_ID);
deleteLocalStorageKey(LOCALSTORAGE.QUICK_FILTERS_SETTINGS_ANNOUNCEMENT);
window.dispatchEvent(new CustomEvent('LOGOUT'));
history.push(ROUTES.LOGIN);
};

View File

@@ -34,7 +34,6 @@ export function mockFieldsValuesAPI(response: {
relatedValues?: (string | null)[];
stringValues?: (string | null)[];
numberValues?: (number | null)[];
boolValues?: (boolean | null)[];
}): void {
server.use(
rest.get('http://localhost/api/v1/fields/values', (_, res, ctx) =>
@@ -47,7 +46,6 @@ export function mockFieldsValuesAPI(response: {
relatedValues: response.relatedValues ?? [],
stringValues: response.stringValues ?? [],
numberValues: response.numberValues ?? [],
boolValues: response.boolValues ?? [],
},
},
}),

View File

@@ -92,12 +92,8 @@ export function useFieldValues({
values.numberValues
?.filter((value): value is number => value !== null && value !== undefined)
.map((value) => value.toString()) || [];
const boolValues =
values.boolValues
?.filter((value): value is boolean => value !== null && value !== undefined)
.map((value) => value.toString()) || [];
return [...stringValues, ...numberValues, ...boolValues];
return [...stringValues, ...numberValues];
}, [data]);
return { relatedValues, allValues, isLoading, isFetching };

View File

@@ -15,30 +15,28 @@ import {
ComboboxTrigger,
} from '@signozhq/ui/combobox';
import { Skeleton, Tooltip } from 'antd';
import { Button } from '@signozhq/ui/button';
import { Switch } from '@signozhq/ui/switch';
import { Typography } from '@signozhq/ui/typography';
import getLocalStorageKey from 'api/browser/localstorage/get';
import setLocalStorageKey from 'api/browser/localstorage/set';
import logEvent from 'api/common/logEvent';
import classNames from 'classnames';
import OverlayScrollbar from 'components/OverlayScrollbar/OverlayScrollbar';
import { LOCALSTORAGE } from 'constants/localStorage';
import { PANEL_TYPES } from 'constants/queryBuilder';
import { useApiMonitoringParams } from 'container/ApiMonitoring/queryParams';
import { useQueryBuilder } from 'hooks/queryBuilder/useQueryBuilder';
import { AuthZGuardContent } from 'lib/authz/components/AuthZGuard/AuthZGuardContent';
import AuthZTooltip from 'lib/authz/components/AuthZTooltip/AuthZTooltip';
import { useAuthZ } from 'lib/authz/hooks/useAuthZ/useAuthZ';
import {
QuickFilterManagePermissions,
QuickFilterReadPermission,
} from 'lib/authz/hooks/useAuthZ/permissions/quick-filter.permissions';
import { isFunction } from 'lodash-es';
import { isFunction, isNull } from 'lodash-es';
import { useAppContext } from 'providers/App/App';
import { Query } from 'types/api/queryBuilder/queryBuilderData';
import { USER_ROLES } from 'types/roles';
import Checkbox from './FilterRenderers/Checkbox/Checkbox';
import CheckboxV2 from './FilterRenderers/Checkbox/v2/CheckboxFilterV2';
import Duration from './FilterRenderers/Duration/Duration';
import Slider from './FilterRenderers/Slider/Slider';
import useFilterConfig from './hooks/useFilterConfig';
import AnnouncementTooltip from './QuickFiltersSettings/AnnouncementTooltip';
import QuickFiltersSettings from './QuickFiltersSettings/QuickFiltersSettings';
import { FiltersType, IQuickFiltersProps, QuickFiltersSource } from './types';
@@ -57,7 +55,9 @@ export default function QuickFilters(props: IQuickFiltersProps): JSX.Element {
showQueryName = true,
useFieldApis,
} = props;
const { user } = useAppContext();
const [isSettingsOpen, setIsSettingsOpen] = useState(false);
const isAdmin = user.role === USER_ROLES.ADMIN;
const [params, setParams] = useApiMonitoringParams();
const showIP = params.showIP ?? true;
@@ -68,12 +68,6 @@ export default function QuickFilters(props: IQuickFiltersProps): JSX.Element {
refetchCustomFilters,
isCustomFiltersLoading,
} = useFilterConfig({ signal, config });
const {
deniedPermissions: deniedSettingsPermissions,
isLoading: isSettingsChecking,
} = useAuthZ(QuickFilterManagePermissions, { enabled: isDynamicFilters });
const isSettingsDisabled =
isSettingsChecking || deniedSettingsPermissions.length > 0;
const {
currentQuery,
@@ -113,6 +107,16 @@ export default function QuickFilters(props: IQuickFiltersProps): JSX.Element {
const shouldShowDropdownInListView =
isListView && source === QuickFiltersSource.TRACES_EXPLORER;
const showAnnouncementTooltip = useMemo(() => {
const localStorageValue = getLocalStorageKey(
LOCALSTORAGE.QUICK_FILTERS_SETTINGS_ANNOUNCEMENT,
);
if (!isNull(localStorageValue)) {
return !(localStorageValue === 'false');
}
return true;
}, []);
const activeQueryIndex = useMemo(() => {
if (isListView) {
return source === QuickFiltersSource.TRACES_EXPLORER
@@ -228,50 +232,49 @@ export default function QuickFilters(props: IQuickFiltersProps): JSX.Element {
const renderRightActions = (): JSX.Element => (
<section className="right-actions">
<Tooltip title="Reset All">
<Button
variant="link"
color="secondary"
aria-label="Reset All"
className="right-action-icon-container"
onClick={handleReset}
prefix={<RefreshCw className="sync-icon" size="md" />}
/>
<div className="right-action-icon-container">
<RefreshCw className="sync-icon" size="md" onClick={handleReset} />
</div>
</Tooltip>
{showFilterCollapse && (
<Tooltip title="Collapse Filters">
<Button
variant="link"
color="secondary"
aria-label="Collapse Filters"
className="right-action-icon-container"
onClick={handleFilterVisibilityChange}
prefix={<ArrowUpToLine style={{ rotate: '270deg' }} size="md" />}
/>
<div className="right-action-icon-container">
<ArrowUpToLine
style={{ rotate: '270deg', cursor: 'pointer' }}
size="md"
onClick={handleFilterVisibilityChange}
/>
</div>
</Tooltip>
)}
{isDynamicFilters && (
<AuthZTooltip checks={QuickFilterManagePermissions}>
<Button
variant="link"
color="secondary"
aria-label="Settings"
{isDynamicFilters && isAdmin && (
<Tooltip title="Settings">
<div
className={classNames('right-action-icon-container', {
active: isSettingsOpen,
})}
onClick={(): void => setIsSettingsOpen(true)}
testId="settings-icon-container"
prefix={
<Tooltip title="Settings" open={isSettingsDisabled ? false : undefined}>
<SettingsIcon
className="settings-icon"
data-testid="settings-icon"
width={14}
height={14}
/>
</Tooltip>
}
/>
</AuthZTooltip>
>
<SettingsIcon
className="settings-icon"
data-testid="settings-icon"
width={14}
height={14}
onClick={(): void => setIsSettingsOpen(true)}
/>
<AnnouncementTooltip
show={showAnnouncementTooltip}
position={{ top: -5, left: 15 }}
title="Edit your quick filters"
message="You can now customize and re-arrange your quick filters panel. Select the quick filters youd need and hide away the rest for faster exploration."
onClose={(): void => {
setLocalStorageKey(
LOCALSTORAGE.QUICK_FILTERS_SETTINGS_ANNOUNCEMENT,
'false',
);
}}
/>
</div>
</Tooltip>
)}
</section>
);
@@ -358,21 +361,6 @@ export default function QuickFilters(props: IQuickFiltersProps): JSX.Element {
</>
);
const filtersSkeleton = (
<div className="quick-filters-skeleton">
{Array.from({ length: 5 }).map((_, index) => (
// eslint-disable-next-line react/no-array-index-key
<Skeleton.Input active size="small" key={index} />
))}
</div>
);
const filtersContent = isCustomFiltersLoading ? (
filtersSkeleton
) : (
<OverlayScrollbar>{renderContent()}</OverlayScrollbar>
);
return (
<div className="quick-filters-container">
<div className="quick-filters">
@@ -383,15 +371,15 @@ export default function QuickFilters(props: IQuickFiltersProps): JSX.Element {
</section>
)}
{signal ? (
<AuthZGuardContent
checks={[QuickFilterReadPermission]}
fallbackOnLoading={filtersSkeleton}
>
{filtersContent}
</AuthZGuardContent>
{isCustomFiltersLoading ? (
<div className="quick-filters-skeleton">
{Array.from({ length: 5 }).map((_, index) => (
// eslint-disable-next-line react/no-array-index-key
<Skeleton.Input active size="small" key={index} />
))}
</div>
) : (
filtersContent
<OverlayScrollbar>{renderContent()}</OverlayScrollbar>
)}
</div>
<div className="quick-filters-settings-container">

View File

@@ -1,202 +0,0 @@
import { ENVIRONMENT } from 'constants/env';
import {
ApiMonitoringParams,
useApiMonitoringParams,
} from 'container/ApiMonitoring/queryParams';
import { useQueryBuilder } from 'hooks/queryBuilder/useQueryBuilder';
import {
QuickFilterReadPermission,
QuickFilterUpdatePermission,
} from 'lib/authz/hooks/useAuthZ/permissions/quick-filter.permissions';
import {
AUTHZ_CHECK_URL,
setupAuthzAdmin,
setupAuthzDeny,
setupAuthzDenyAll,
} from 'lib/authz/utils/authz-test-utils';
import {
otherFiltersResponse,
quickFiltersAttributeValuesResponse,
quickFiltersListResponse,
} from 'mocks-server/__mockdata__/customQuickFilters';
import { server } from 'mocks-server/server';
import { rest } from 'msw';
import { render, screen, userEvent, waitFor } from 'tests/test-utils';
import QuickFilters from '../QuickFilters';
import { QuickFiltersSource, SignalType } from '../types';
import { QuickFiltersConfig } from './constants';
jest.mock('hooks/queryBuilder/useQueryBuilder', () => ({
useQueryBuilder: jest.fn(),
}));
jest.mock('container/ApiMonitoring/queryParams');
const mockUseApiMonitoringParams = jest.mocked(useApiMonitoringParams);
const BASE_URL = ENVIRONMENT.baseURL;
const SIGNAL = SignalType.LOGS;
const quickFiltersListURL = `${BASE_URL}/api/v2/quick_filters/${SIGNAL}`;
const fieldsKeysURL = `${BASE_URL}/api/v1/fields/keys`;
const attributeValuesURL = `${BASE_URL}/api/v3/autocomplete/attribute_values`;
const fieldsValuesURL = `${BASE_URL}/api/v1/fields/values`;
const NOT_AUTHORIZED_TEXT = /is not authorized to perform/i;
const FILTER_SERVICE_NAME = 'Service Name';
const SETTINGS_CONTAINER_TEST_ID = 'settings-icon-container';
beforeEach(() => {
(useQueryBuilder as jest.Mock).mockReturnValue({
currentQuery: {
builder: {
queryData: [
{
queryName: 'Test Query',
filters: { items: [] },
},
],
},
},
lastUsedQuery: 0,
redirectWithQueryBuilderData: jest.fn(),
});
mockUseApiMonitoringParams.mockReturnValue([
{ showIP: true } as ApiMonitoringParams,
jest.fn(),
]);
server.use(
rest.get(quickFiltersListURL, (_req, res, ctx) =>
res(ctx.status(200), ctx.json(quickFiltersListResponse)),
),
rest.get(fieldsKeysURL, (_req, res, ctx) =>
res(ctx.status(200), ctx.json(otherFiltersResponse)),
),
rest.get(attributeValuesURL, (_req, res, ctx) =>
res(ctx.status(200), ctx.json(quickFiltersAttributeValuesResponse)),
),
rest.get(fieldsValuesURL, (_req, res, ctx) =>
res(ctx.status(200), ctx.json(quickFiltersAttributeValuesResponse)),
),
);
});
afterEach(() => {
server.resetHandlers();
jest.clearAllMocks();
});
function renderWithSignal(): void {
render(
<QuickFilters
source={QuickFiltersSource.LOGS_EXPLORER}
signal={SIGNAL}
handleFilterVisibilityChange={jest.fn()}
/>,
);
}
function renderStaticConfig(): void {
render(
<QuickFilters
source={QuickFiltersSource.EXCEPTIONS}
config={QuickFiltersConfig}
handleFilterVisibilityChange={jest.fn()}
/>,
);
}
describe('QuickFilters - AuthZ', () => {
describe('read denied', () => {
it('shows the inline denial instead of the filters, header stays', async () => {
server.use(setupAuthzDeny(QuickFilterReadPermission));
renderWithSignal();
await expect(
screen.findByText(NOT_AUTHORIZED_TEXT),
).resolves.toBeInTheDocument();
expect(screen.queryByText(FILTER_SERVICE_NAME)).not.toBeInTheDocument();
expect(screen.getByText('Filters for')).toBeInTheDocument();
});
});
describe('update denied', () => {
it('renders the filters but disables the settings trigger', async () => {
server.use(setupAuthzDeny(QuickFilterUpdatePermission));
renderWithSignal();
await expect(
screen.findByText(FILTER_SERVICE_NAME),
).resolves.toBeInTheDocument();
const settingsTrigger = await screen.findByTestId(
SETTINGS_CONTAINER_TEST_ID,
);
await waitFor(() =>
expect(settingsTrigger).toHaveAttribute(
'data-denied-permissions',
expect.stringContaining('update'),
),
);
await userEvent.click(settingsTrigger);
expect(screen.queryByText(/ADDED FILTERS/i)).not.toBeInTheDocument();
});
});
describe('all permissions granted', () => {
it('renders the filters and opens settings from the trigger', async () => {
server.use(setupAuthzAdmin());
renderWithSignal();
await expect(
screen.findByText(FILTER_SERVICE_NAME),
).resolves.toBeInTheDocument();
const settingsTrigger = await screen.findByTestId(
SETTINGS_CONTAINER_TEST_ID,
);
expect(settingsTrigger).not.toHaveAttribute('data-denied-permissions');
await userEvent.click(settingsTrigger);
await expect(
screen.findByText(/ADDED FILTERS/i),
).resolves.toBeInTheDocument();
});
});
describe('static config pages (no signal)', () => {
it('is not gated even when every permission is denied', async () => {
server.use(setupAuthzDenyAll());
renderStaticConfig();
await expect(
screen.findByText(FILTER_SERVICE_NAME),
).resolves.toBeInTheDocument();
expect(screen.queryByText(NOT_AUTHORIZED_TEXT)).not.toBeInTheDocument();
});
});
describe('permission check loading', () => {
it('shows the skeleton, not the filters or a denial', async () => {
server.use(
rest.post(AUTHZ_CHECK_URL, (_req, res, ctx) => res(ctx.delay('infinite'))),
);
renderWithSignal();
await waitFor(() =>
// eslint-disable-next-line testing-library/no-node-access
expect(
document.querySelector('.quick-filters-skeleton'),
).toBeInTheDocument(),
);
expect(screen.queryByText(NOT_AUTHORIZED_TEXT)).not.toBeInTheDocument();
expect(screen.queryByText(FILTER_SERVICE_NAME)).not.toBeInTheDocument();
});
});
});

View File

@@ -8,7 +8,6 @@ import { FiltersType, IQuickFiltersConfig, SignalType } from './types';
const FILTER_TITLE_MAP: Record<string, string> = {
duration_nano: 'Duration',
hasError: 'Has Error (Status)',
has_error: 'Has Error (Status)',
};
const FILTER_TYPE_MAP: Record<string, FiltersType> = {

View File

@@ -31,6 +31,7 @@ export enum LOCALSTORAGE {
DONT_SHOW_SLOW_API_WARNING = 'DONT_SHOW_SLOW_API_WARNING',
METRICS_LIST_OPTIONS = 'METRICS_LIST_OPTIONS',
SHOW_EXCEPTIONS_QUICK_FILTERS = 'SHOW_EXCEPTIONS_QUICK_FILTERS',
QUICK_FILTERS_SETTINGS_ANNOUNCEMENT = 'QUICK_FILTERS_SETTINGS_ANNOUNCEMENT',
FUNNEL_STEPS = 'FUNNEL_STEPS',
SPAN_DETAILS_PINNED_ATTRIBUTES = 'SPAN_DETAILS_PINNED_ATTRIBUTES',
LAST_USED_CUSTOM_TIME_RANGES = 'LAST_USED_CUSTOM_TIME_RANGES',

View File

@@ -184,11 +184,6 @@
text-decoration-thickness: 2px;
text-underline-offset: 2px;
}
&[disabled] {
opacity: 0.6;
text-decoration: none;
}
}
.workspace-restricted-banner,

View File

@@ -16,6 +16,7 @@ import * as Sentry from '@sentry/react';
import { Toaster } from '@signozhq/ui/sonner';
import { TooltipProvider } from '@signozhq/ui/tooltip';
import { Flex } from 'antd';
import { Button } from '@signozhq/ui/button';
import getLocalStorageApi from 'api/browser/localstorage/get';
import setLocalStorageApi from 'api/browser/localstorage/set';
import getChangelogByVersion from 'api/changelog/getChangelogByVersion';
@@ -819,9 +820,14 @@ function AppLayout(props: AppLayoutProps): JSX.Element {
{' '}
Please{' '}
<AuthZTooltip checks={SubscriptionManagePermissions}>
<a className="upgrade-link" onClick={handleFailedPayment}>
<Button
variant="link"
color="none"
className="upgrade-link"
onClick={handleFailedPayment}
>
pay the bill
</a>
</Button>
</AuthZTooltip>
to continue using SigNoz features.
<span className="refresh-payment-status">

View File

@@ -60,10 +60,6 @@
margin: var(--spacing-12) var(--spacing-4);
}
.usageDenied {
margin-bottom: var(--spacing-4);
}
.billingDetails {
margin: var(--spacing-4) 0;
border: 1px solid var(--l1-border);

View File

@@ -37,7 +37,6 @@ import { isEmpty, pick } from 'lodash-es';
import AuthZButton from 'lib/authz/components/AuthZButton/AuthZButton';
import AuthZTooltip from 'lib/authz/components/AuthZTooltip/AuthZTooltip';
import { AuthZGuardContent } from 'lib/authz/components/AuthZGuard/AuthZGuardContent';
import PermissionDeniedCallout from 'lib/authz/components/PermissionDeniedCallout/PermissionDeniedCallout';
import {
SubscriptionCreatePermission,
SubscriptionManagePermissions,
@@ -510,15 +509,7 @@ export default function BillingContainer(): JSX.Element {
))}
</Card>
<AuthZGuardContent
checks={[SubscriptionReadPermission]}
fallback={({ deniedPermissions }): JSX.Element => (
<PermissionDeniedCallout
deniedPermissions={deniedPermissions}
className={styles.usageDenied}
/>
)}
>
<AuthZGuardContent checks={[SubscriptionReadPermission]}>
<>
<div className={styles.billingGraphSection}>
{!isLoading && !isFetchingBillingData ? (

View File

@@ -1,14 +0,0 @@
.container {
display: flex;
align-items: center;
justify-content: flex-end;
gap: 0.3rem;
margin: 8px 0;
}
.optionsTrigger {
display: flex;
align-items: center;
gap: 4px;
cursor: pointer;
}

View File

@@ -1,82 +0,0 @@
import { memo, useState } from 'react';
import { useTranslation } from 'react-i18next';
import { Settings } from '@signozhq/icons';
import FieldsSelector from 'components/FieldsSelector';
import Controls, { ControlsProps } from 'container/Controls';
import { OptionsMenuConfig } from 'container/OptionsMenu/types';
import useQueryPagination from 'hooks/queryPagination/useQueryPagination';
import { DataSource } from 'types/common/queryBuilder';
import styles from './Controls.module.scss';
function TraceExplorerControls({
isLoading,
totalCount,
perPageOptions,
config,
showSizeChanger = true,
}: TraceExplorerControlsProps): JSX.Element | null {
const { t } = useTranslation(['trace']);
const [isFieldsSelectorOpen, setIsFieldsSelectorOpen] = useState(false);
const {
pagination,
handleCountItemsPerPageChange,
handleNavigateNext,
handleNavigatePrevious,
} = useQueryPagination(totalCount, perPageOptions);
return (
<div className={styles.container}>
{config?.fieldsSelector && (
<>
<div
className={styles.optionsTrigger}
onClick={(): void => setIsFieldsSelectorOpen(true)}
>
{t('options_menu.options')}
<Settings size="md" />
</div>
<FieldsSelector
isOpen={isFieldsSelectorOpen}
title="Edit columns"
fields={config.fieldsSelector.value}
onFieldsChange={config.fieldsSelector.onFieldsChange}
onClose={(): void => setIsFieldsSelectorOpen(false)}
signal={DataSource.TRACES}
/>
</>
)}
<Controls
isLoading={isLoading}
totalCount={totalCount}
offset={pagination.offset}
countPerPage={pagination.limit}
perPageOptions={perPageOptions}
handleCountItemsPerPageChange={handleCountItemsPerPageChange}
handleNavigateNext={handleNavigateNext}
handleNavigatePrevious={handleNavigatePrevious}
showSizeChanger={showSizeChanger}
/>
</div>
);
}
TraceExplorerControls.defaultProps = {
config: null,
};
type TraceExplorerControlsProps = Pick<
ControlsProps,
'isLoading' | 'totalCount' | 'perPageOptions'
> & {
config?: OptionsMenuConfig | null;
showSizeChanger?: boolean;
};
TraceExplorerControls.defaultProps = {
showSizeChanger: true,
};
export default memo(TraceExplorerControls);

View File

@@ -1,168 +0,0 @@
import { Link } from 'react-router-dom';
import type { TableColumnsType as ColumnsType } from 'antd';
import { Badge } from '@signozhq/ui/badge';
import { Typography } from '@signozhq/ui/typography';
import { TelemetryFieldKey } from 'api/v5/v5';
import type { TracesTableRow } from '../TracesTable/getFieldColumn';
import { DATE_TIME_FORMATS } from 'constants/dateTimeFormats';
import ROUTES from 'constants/routes';
import { buildCompositeKey } from 'container/OptionsMenu/utils';
import { getMs } from 'container/Trace/Filters/Panel/PanelBody/Duration/util';
import { formUrlParams } from 'container/TraceDetail/utils';
import { TimestampInput } from 'hooks/useTimezoneFormatter/useTimezoneFormatter';
import { RowData } from 'lib/query/createTableColumnsFromQuery';
import LineClampedText from 'periscope/components/LineClampedText/LineClampedText';
import { ILog } from 'types/api/logs/log';
import { QueryDataV3 } from 'types/api/widgets/getQuery';
export function BlockLink({
children,
to,
openInNewTab,
}: {
children: React.ReactNode;
to: string;
openInNewTab: boolean;
}): any {
// Display block to make the whole cell clickable
return (
<Link
to={to}
style={{ display: 'block' }}
target={openInNewTab ? '_blank' : '_self'}
>
{children}
</Link>
);
}
export const transformDataWithDate = (
data: QueryDataV3[],
): Omit<ILog, 'timestamp'>[] =>
data[0]?.list?.map(({ data, timestamp }) => ({ ...data, date: timestamp })) ||
[];
export const getTraceLink = (record: Record<string, unknown>): string => {
function readId(value: unknown): string {
if (typeof value === 'string' || typeof value === 'number') {
return String(value);
}
return '';
}
const traceId = readId(record.traceID) || readId(record.trace_id);
const spanId = readId(record.spanID) || readId(record.span_id);
return `${ROUTES.TRACE}/${traceId}${formUrlParams({
spanId,
levelUp: 0,
levelDown: 0,
})}`;
};
export const getListColumns = (
selectedColumns: TelemetryFieldKey[],
formatTimezoneAdjustedTimestamp: (
input: TimestampInput,
format?: string,
) => string | number,
): ColumnsType<RowData> => {
const initialColumns: ColumnsType<RowData> = [
{
dataIndex: 'date',
key: 'date',
title: 'Timestamp',
width: 145,
render: (value, item): JSX.Element => {
const date =
typeof value === 'string'
? formatTimezoneAdjustedTimestamp(
value,
DATE_TIME_FORMATS.ISO_DATETIME_MS,
)
: formatTimezoneAdjustedTimestamp(
value / 1e6,
DATE_TIME_FORMATS.ISO_DATETIME_MS,
);
return (
<BlockLink to={getTraceLink(item)} openInNewTab={false}>
<Typography.Text>{date}</Typography.Text>
</BlockLink>
);
},
},
];
const columns: ColumnsType<RowData> =
selectedColumns.map((props) => {
const name = props?.name || (props as any)?.key;
const fieldContext = props?.fieldContext || (props as any)?.type;
return {
title: name,
dataIndex: name,
key: buildCompositeKey(name, fieldContext),
width: 145,
render: (value, item): JSX.Element => {
if (value === '') {
return (
<BlockLink to={getTraceLink(item)} openInNewTab={false}>
<Typography data-testid={name}>N/A</Typography>
</BlockLink>
);
}
if (
name === 'httpMethod' ||
name === 'responseStatusCode' ||
name === 'response_status_code' ||
name === 'http_method'
) {
return (
<BlockLink to={getTraceLink(item)} openInNewTab={false}>
<Badge data-testid={name} color="sakura" variant="outline">
{value}
</Badge>
</BlockLink>
);
}
if (name === 'durationNano' || name === 'duration_nano') {
return (
<BlockLink to={getTraceLink(item)} openInNewTab={false}>
<Typography data-testid={name}>{getMs(value)}ms</Typography>
</BlockLink>
);
}
return (
<BlockLink to={getTraceLink(item)} openInNewTab={false}>
<Typography data-testid={name}>
<LineClampedText text={value} lines={3} />
</Typography>
</BlockLink>
);
},
responsive: ['md'],
};
}) || [];
return [...initialColumns, ...columns];
};
// Reshapes the query-range list payload into table rows. `id` mirrors span_id so
// TanStack sees genuine row changes on orderBy toggles instead of falling back to
// positional ids; `timestamp` is lifted from the wrapping ListItem.
export const transformSpanRows = (data: QueryDataV3[]): TracesTableRow[] => {
const list = data[0]?.list;
if (!list) {
return [];
}
return list.map((item) => {
const row = item.data as Record<string, unknown>;
return {
...row,
timestamp: item.timestamp,
id: row.span_id,
};
}) as TracesTableRow[];
};

View File

@@ -1,19 +0,0 @@
.loading-traces {
padding: 24px 0;
height: 240px;
display: flex;
justify-content: center;
align-items: flex-start;
.loading-traces-content {
display: flex;
align-items: flex-start;
flex-direction: column;
.loading-gif {
height: 72px;
margin-left: -24px;
}
}
}

View File

@@ -1,22 +0,0 @@
import { useTranslation } from 'react-i18next';
import { Typography } from '@signozhq/ui/typography';
import { DataSource } from 'types/common/queryBuilder';
import loadingPlaneUrl from '@/assets/Icons/loading-plane.gif';
import './TraceLoading.styles.scss';
export function TracesLoading(): JSX.Element {
const { t } = useTranslation('common');
return (
<div className="loading-traces">
<div className="loading-traces-content">
<img className="loading-gif" src={loadingPlaneUrl} alt="wait-icon" />
<Typography>
{t('pending_data_placeholder', { dataSource: DataSource.TRACES })}
</Typography>
</div>
</div>
);
}

View File

@@ -1,77 +0,0 @@
import { generatePath, Link } from 'react-router-dom';
import { Badge } from '@signozhq/ui/badge';
import TanStackTable from 'components/TanStackTableView';
import { DATE_TIME_FORMATS } from 'constants/dateTimeFormats';
import ROUTES from 'constants/routes';
import { getMs } from 'container/Trace/Filters/Panel/PanelBody/Duration/util';
import { useTimezone } from 'providers/Timezone';
import {
DURATION_FIELD_NAMES,
STATUS_FIELD_NAMES,
TIMESTAMP_FIELD_NAMES,
TRACE_ID_FIELD_NAMES,
} from './constants';
import { stringifyCellValue } from './utils';
type FieldCellProps = {
name: string;
value: unknown;
};
function FieldCell({ name, value }: FieldCellProps): JSX.Element {
const { formatTimezoneAdjustedTimestamp } = useTimezone();
if (TIMESTAMP_FIELD_NAMES.has(name)) {
const ts = value as string | number;
const formatted =
typeof ts === 'string'
? formatTimezoneAdjustedTimestamp(ts, DATE_TIME_FORMATS.ISO_DATETIME_MS)
: formatTimezoneAdjustedTimestamp(
ts / 1e6,
DATE_TIME_FORMATS.ISO_DATETIME_MS,
);
const text = String(formatted);
return <TanStackTable.Text title={text}>{text}</TanStackTable.Text>;
}
if (value === '' || value == null) {
return <TanStackTable.Text data-testid={name}>-</TanStackTable.Text>;
}
const text = stringifyCellValue(value);
if (TRACE_ID_FIELD_NAMES.has(name)) {
return (
<Link
to={generatePath(ROUTES.TRACE_DETAIL, { id: text })}
data-testid="trace-id"
onClick={(e): void => e.stopPropagation()}
>
{text}
</Link>
);
}
if (STATUS_FIELD_NAMES.has(name)) {
return (
<Badge data-testid={name} color="sakura" variant="outline">
{text}
</Badge>
);
}
if (DURATION_FIELD_NAMES.has(name)) {
return (
<TanStackTable.Text data-testid={name}>{getMs(text)}ms</TanStackTable.Text>
);
}
return (
<TanStackTable.Text data-testid={name} title={text}>
{text}
</TanStackTable.Text>
);
}
export default FieldCell;

View File

@@ -1,26 +0,0 @@
.tableWrapper {
flex: 1;
min-height: 0;
display: flex;
flex-direction: column;
}
.tracesTable {
--tanstack-table-row-height: 54px;
--tanstack-table-header-height: 54px;
--tanstack-cell-padding-top-override: 5px;
--tanstack-cell-padding-bottom-override: 5px;
--tanstack-cell-padding-right-override: 15px;
--tanstack-cell-padding-left-override: 15px;
--tanstack-cell-header-padding-left-override: 5px;
--tanstack-cell-header-padding-left-first-column: 15px;
--tanstack-plain-body-line-clamp: 1;
--tanstack-table-cell-bg: var(--l2-background);
--tanstack-table-header-cell-bg: var(--l1-background-hover);
--tanstack-table-row-hover-bg: var(--l1-background-hover);
}

View File

@@ -1,116 +0,0 @@
import { useCallback } from 'react';
import { useHistory } from 'react-router-dom';
import ErrorInPlace from 'components/ErrorInPlace/ErrorInPlace';
import TanStackTable from 'components/TanStackTableView';
import type {
CellTypographySize,
TableColumnDef,
} from 'components/TanStackTableView/types';
import EmptyLogsSearch from 'container/EmptyLogsSearch/EmptyLogsSearch';
import NoLogs from 'container/NoLogs/NoLogs';
import { TracesLoading } from '../TraceLoading/TraceLoading';
import APIError from 'types/api/error';
import { DataSource, PanelTypeKeys } from 'types/common/queryBuilder';
import { getAbsoluteUrl } from 'utils/basePath';
import type { TracesTableRow } from './getFieldColumn';
import styles from './TracesTable.module.scss';
export type TracesTableProps = {
data: TracesTableRow[];
columns: TableColumnDef<TracesTableRow>[];
columnStorageKey?: string;
respectColumnOrder?: boolean;
panelType: PanelTypeKeys;
/** Builds the trace-detail href for a row; drives row click + cmd/ctrl-click. */
getRowHref: (row: TracesTableRow) => string;
isLoading: boolean;
isFetching: boolean;
isError: boolean;
error: APIError | Error | null;
isFilterApplied: boolean;
onColumnOrderChange?: (cols: TableColumnDef<TracesTableRow>[]) => void;
onColumnRemove?: (columnId: string) => void;
cellTypographySize?: CellTypographySize;
};
function TracesTable({
data,
columns,
columnStorageKey,
respectColumnOrder = false,
panelType,
getRowHref,
isLoading,
isFetching,
isError,
error,
isFilterApplied,
onColumnOrderChange,
onColumnRemove,
cellTypographySize = 'medium',
}: TracesTableProps): JSX.Element {
const history = useHistory();
const isDataAbsent =
!isLoading && !isFetching && !isError && data.length === 0;
const handleRowClick = useCallback(
(row: TracesTableRow): void => {
history.push(getRowHref(row));
},
[history, getRowHref],
);
const handleRowClickNewTab = useCallback(
(row: TracesTableRow): void => {
window.open(getAbsoluteUrl(getRowHref(row)), '_blank', 'noopener');
},
[getRowHref],
);
return (
<>
{isError && error && <ErrorInPlace error={error as APIError} />}
{(isLoading || (isFetching && data.length === 0)) && <TracesLoading />}
{isDataAbsent && !isFilterApplied && (
<NoLogs dataSource={DataSource.TRACES} />
)}
{isDataAbsent && isFilterApplied && (
<EmptyLogsSearch dataSource={DataSource.TRACES} panelType={panelType} />
)}
{!isError && data.length !== 0 && (
<div className={styles.tableWrapper}>
<TanStackTable<TracesTableRow>
data={data}
columns={columns}
className={styles.tracesTable}
columnStorageKey={columnStorageKey}
respectColumnOrder={respectColumnOrder}
isLoading={isFetching}
cellTypographySize={cellTypographySize}
onColumnOrderChange={onColumnOrderChange}
onColumnRemove={onColumnRemove}
onRowClick={handleRowClick}
onRowClickNewTab={handleRowClickNewTab}
getRowTestId={(row): string => `traces-table-row-${row.id}`}
/>
</div>
)}
</>
);
}
TracesTable.defaultProps = {
columnStorageKey: undefined,
respectColumnOrder: false,
onColumnOrderChange: undefined,
onColumnRemove: undefined,
cellTypographySize: 'medium',
};
export default TracesTable;

View File

@@ -1,18 +0,0 @@
// Field-name allowlists that drive signal-specific cell rendering. Both legacy
// camelCase and snake_case variants are listed because the API has shipped both.
export const TIMESTAMP_FIELD_NAMES = new Set(['timestamp']);
export const STATUS_FIELD_NAMES = new Set([
'httpMethod',
'http_method',
'http.method',
'http.request.method',
'responseStatusCode',
'response_status_code',
'http.status_code',
'http.response.status_code',
]);
export const DURATION_FIELD_NAMES = new Set(['durationNano', 'duration_nano']);
export const TRACE_ID_FIELD_NAMES = new Set(['traceID', 'trace_id']);

View File

@@ -1,26 +0,0 @@
import { TelemetryFieldKey } from 'api/v5/v5';
import type { TableColumnDef } from 'components/TanStackTableView/types';
import { buildCompositeKey } from 'container/OptionsMenu/utils';
import { TIMESTAMP_FIELD_NAMES } from './constants';
import FieldCell from './FieldCell';
export type TracesTableRow = { id: string } & Record<string, unknown>;
export function getFieldColumn(
field: TelemetryFieldKey,
): TableColumnDef<TracesTableRow> {
const { name, fieldContext, fieldDataType } = field;
const isTimestamp = TIMESTAMP_FIELD_NAMES.has(name);
return {
id: buildCompositeKey(name, fieldContext, fieldDataType),
header: name,
accessorFn: (row): unknown => row[name],
enableMove: !isTimestamp,
enableRemove: !isTimestamp,
canBeHidden: !isTimestamp,
width: { min: 192 },
cell: ({ value }): JSX.Element => <FieldCell name={name} value={value} />,
};
}

View File

@@ -1,12 +0,0 @@
export function stringifyCellValue(value: unknown): string {
if (value == null) {
return '';
}
if (typeof value === 'string') {
return value;
}
if (typeof value === 'number' || typeof value === 'boolean') {
return String(value);
}
return JSON.stringify(value);
}

View File

@@ -1,6 +1,9 @@
import { TelemetryFieldKey } from 'api/v5/v5';
import type { TableColumnDef } from 'components/TanStackTableView/types';
import { getFieldColumn, TracesTableRow } from '../TracesTable/getFieldColumn';
import {
getFieldColumn,
TracesTableRow,
} from 'container/TracesExplorer/TracesTable/getFieldColumn';
import { DEFAULT_PER_PAGE_OPTIONS } from 'hooks/queryPagination';
export const PER_PAGE_OPTIONS: number[] = [10, ...DEFAULT_PER_PAGE_OPTIONS];

View File

@@ -1,235 +0,0 @@
/**
* AI Assistant page-action factories for the Traces Explorer.
*
* Mirrors the logs equivalents — each factory closes over live page
* state/callbacks so `execute()` always operates on the current query, and
* the page component instantiates them via `useMemo` + `usePageActions`.
*
* See `pages/LogsExplorer/aiActions.ts` for the rationale behind writing
* BOTH `filters.items` and `filter.expression` and then re-using the same
* URL parser shape via `redirectWithQueryBuilderData`.
*/
import { convertFiltersToExpression } from 'components/QueryBuilderV2/utils';
import {
aiFilterToTagFilterItem,
FILTER_OP_ENUM,
FILTER_VALUE_DESCRIPTION,
FilterDeps,
replaceFirstQueryData,
} from 'container/AIAssistant/pageActions/builderQueryHelpers';
import {
ActionResult,
PageAction,
} from 'container/AIAssistant/pageActions/types';
import {
IBuilderQuery,
TagFilterItem,
} from 'types/api/queryBuilder/queryBuilderData';
interface AIFilter {
key: string;
op: string;
value: string;
}
interface RunQueryParams {
filters: AIFilter[];
}
interface AddFilterParams {
key: string;
op: string;
value: string;
}
type TracesView = 'list' | 'timeseries' | 'table' | 'trace';
interface ChangeViewParams {
view: TracesView;
}
interface SaveViewParams {
name: string;
}
/**
* Replace all active span filters and navigate to the updated query URL
* (which makes the WHERE clause reflect the new filters and triggers a re-run).
*/
export function tracesRunQueryAction(
deps: FilterDeps,
): PageAction<RunQueryParams> {
return {
id: 'traces.runQuery',
description: 'Replace the active trace filters and re-run the query',
parameters: {
type: 'object',
properties: {
filters: {
type: 'array',
description: 'Replacement filter list',
items: {
type: 'object',
properties: {
key: {
type: 'string',
description: 'Attribute key, e.g. service.name, http.status_code',
},
op: {
type: 'string',
enum: [...FILTER_OP_ENUM],
},
value: {
type: 'string',
description: FILTER_VALUE_DESCRIPTION,
},
},
required: ['key', 'op', 'value'],
},
},
},
required: ['filters'],
},
autoApply: true,
execute: async ({ filters }): Promise<ActionResult> => {
const baseQuery = deps.currentQuery.builder.queryData[0];
if (!baseQuery) {
throw new Error('No active query found in Traces Explorer.');
}
const tagItems = filters.map(aiFilterToTagFilterItem);
const newFilters = { items: tagItems, op: 'AND' };
const updatedBuilderQuery: IBuilderQuery = {
...baseQuery,
filters: newFilters,
filter: convertFiltersToExpression(newFilters),
};
deps.handleSetQueryData(0, updatedBuilderQuery);
deps.redirectWithQueryBuilderData(
replaceFirstQueryData(deps.currentQuery, updatedBuilderQuery),
);
return {
summary: `Query updated with ${filters.length} filter(s) and re-run.`,
};
},
getContext: (): Record<string, unknown> => ({
filters:
deps.currentQuery.builder.queryData[0]?.filters?.items?.map(
(f: TagFilterItem) => ({
key: f.key?.key,
op: f.op,
value: f.value,
}),
) ?? [],
}),
};
}
/**
* Append a single filter to the existing trace query and navigate to the
* updated URL.
*/
export function tracesAddFilterAction(
deps: FilterDeps,
): PageAction<AddFilterParams> {
return {
id: 'traces.addFilter',
description: 'Add a single filter to the current trace query and re-run',
parameters: {
type: 'object',
properties: {
key: {
type: 'string',
description: 'Attribute key, e.g. service.name, http.status_code',
},
op: {
type: 'string',
enum: [...FILTER_OP_ENUM],
},
value: {
type: 'string',
description: FILTER_VALUE_DESCRIPTION,
},
},
required: ['key', 'op', 'value'],
},
autoApply: true,
execute: async ({ key, op, value }): Promise<ActionResult> => {
const baseQuery = deps.currentQuery.builder.queryData[0];
if (!baseQuery) {
throw new Error('No active query found in Traces Explorer.');
}
const existing = baseQuery.filters?.items ?? [];
const newItem = aiFilterToTagFilterItem({ key, op, value });
const newFilters = { items: [...existing, newItem], op: 'AND' };
const updatedBuilderQuery: IBuilderQuery = {
...baseQuery,
filters: newFilters,
filter: convertFiltersToExpression(newFilters),
};
deps.handleSetQueryData(0, updatedBuilderQuery);
deps.redirectWithQueryBuilderData(
replaceFirstQueryData(deps.currentQuery, updatedBuilderQuery),
);
return { summary: `Filter added: ${key} ${op} "${value}". Query re-run.` };
},
};
}
/**
* Switch the traces explorer between list / timeseries / table / trace views.
*/
export function tracesChangeViewAction(deps: {
onChangeView: (view: TracesView) => void;
}): PageAction<ChangeViewParams> {
return {
id: 'traces.changeView',
description:
'Switch the Traces Explorer between list, timeseries, table, and trace views',
parameters: {
type: 'object',
properties: {
view: {
type: 'string',
enum: ['list', 'timeseries', 'table', 'trace'],
description: 'The panel view to switch to',
},
},
required: ['view'],
},
execute: async ({ view }): Promise<ActionResult> => {
deps.onChangeView(view);
return { summary: `Switched to the "${view}" view.` };
},
};
}
/**
* Save the current trace query as a named view (stub — wires to real API
* when available).
*/
export function tracesSaveViewAction(deps: {
onSaveView: (name: string) => Promise<void>;
}): PageAction<SaveViewParams> {
return {
id: 'traces.saveView',
description: 'Save the current trace query as a named view',
parameters: {
type: 'object',
properties: {
name: { type: 'string', description: 'Name for the saved view' },
},
required: ['name'],
},
execute: async ({ name }): Promise<ActionResult> => {
await deps.onSaveView(name);
return { summary: `View "${name}" saved.` };
},
};
}

View File

@@ -1,132 +0,0 @@
import {
ArrowUpToLine,
Atom,
Filter,
SquareMousePointer,
Terminal,
Binoculars,
} from '@signozhq/icons';
import { Button, Tooltip } from 'antd';
import cx from 'classnames';
import { ExplorerViews } from 'pages/LogsExplorer/utils';
import './ToolbarActions.styles.scss';
interface LeftToolbarActionsProps {
items: any;
selectedView: string;
onChangeSelectedView: (view: ExplorerViews) => void;
showFilter: boolean;
handleFilterVisibilityChange: () => void;
}
const activeTab = 'active-tab';
export default function LeftToolbarActions({
items,
selectedView,
onChangeSelectedView,
showFilter,
handleFilterVisibilityChange,
}: LeftToolbarActionsProps): JSX.Element {
const { clickhouse, list, timeseries, table, trace } = items;
return (
<div className="left-toolbar">
{!showFilter && (
<Tooltip title="Show Filters">
<Button onClick={handleFilterVisibilityChange} className="filter-btn">
<Filter size={12} />
<ArrowUpToLine size={12} style={{ transform: 'rotate(90deg)' }} />
</Button>
</Tooltip>
)}
<div className="left-toolbar-query-actions">
{list?.show && (
<Tooltip title="List View">
<Button
disabled={list.disabled}
className={cx(
'list-view-tab',
'explorer-view-option',
selectedView === list.key ? activeTab : '',
)}
onClick={(): void => onChangeSelectedView(list.key)}
>
<SquareMousePointer size={14} data-testid="search-view" />
List View
</Button>
</Tooltip>
)}
{trace?.show && (
<Tooltip title="Trace View">
<Button
disabled={trace.disabled}
className={cx(
'trace-view-tab',
'explorer-view-option',
selectedView === trace.key ? activeTab : '',
)}
onClick={(): void => onChangeSelectedView(trace.key)}
>
<SquareMousePointer size={14} data-testid="trace-view" />
Trace View
</Button>
</Tooltip>
)}
{timeseries?.show && (
<Tooltip title="Time Series">
<Button
disabled={timeseries.disabled}
className={cx(
'timeseries-view-tab',
'explorer-view-option',
selectedView === timeseries.key ? activeTab : '',
)}
onClick={(): void => onChangeSelectedView(timeseries.key)}
>
<Atom size={14} data-testid="query-builder-view" />
Time Series
</Button>
</Tooltip>
)}
{clickhouse?.show && (
<Tooltip title="Clickhouse">
<Button
disabled={clickhouse.disabled}
className={cx(
'clickhouse-view-tab',
'explorer-view-option',
selectedView === clickhouse.key ? activeTab : '',
)}
onClick={(): void => onChangeSelectedView(clickhouse.key)}
>
<Terminal size={14} data-testid="clickhouse-view" />
Clickhouse
</Button>
</Tooltip>
)}
{table?.show && (
<Tooltip title="Table">
<Button
disabled={table.disabled}
className={cx(
'table-view-tab',
'explorer-view-option',
selectedView === table.key ? activeTab : '',
)}
onClick={(): void => onChangeSelectedView(table.key)}
>
<Binoculars size={14} data-testid="query-builder-view-v2" />
Table
</Button>
</Tooltip>
)}
</div>
</div>
);
}

View File

@@ -1,125 +0,0 @@
.left-toolbar {
display: flex;
align-items: center;
.filter-btn {
display: flex;
align-items: center;
justify-content: center;
box-shadow: none;
height: 32px;
margin-right: 12px;
border: 1px solid var(--l1-border);
}
.left-toolbar-query-actions {
display: flex;
border-radius: 2px;
border: 1px solid var(--l1-border);
background: var(--l1-background);
flex-direction: row;
border-bottom: none;
margin-bottom: -1px;
.prom-ql-icon {
height: 14px;
width: 14px;
}
.explorer-view-option {
display: flex;
align-items: center;
justify-content: center;
flex-direction: row;
border: none;
padding: 9px;
box-shadow: none;
border-radius: 0px;
border-left: 1px solid var(--l1-border);
border-bottom: 1px solid var(--l1-border);
gap: 8px;
&.active-tab {
background-color: var(--primary-background);
border-bottom: 1px solid var(--primary-background);
color: var(--primary-foreground);
&:hover {
background-color: var(--primary-background) !important;
}
}
&:disabled {
background-color: var(--l3-background);
opacity: 0.6;
}
&:first-child {
border-left: 1px solid transparent;
}
&:hover {
background-color: transparent !important;
border-left: 1px solid transparent !important;
color: var(--l1-foreground);
}
}
}
.frequency-chart-view-controller {
display: flex;
align-items: center;
padding-left: 8px;
gap: 8px;
}
}
.right-toolbar {
display: flex;
align-items: center;
background-color: var(--bg-robin-600);
}
.right-actions {
display: flex;
align-items: center;
}
.loading-container {
display: flex;
gap: 8px;
align-items: center;
.loading-btn {
display: flex;
width: 32px;
height: 33px;
padding: 4px 10px;
justify-content: center;
align-items: center;
gap: 6px;
flex-shrink: 0;
border-radius: 2px;
background: var(--l3-background);
box-shadow: none;
border: none;
}
.cancel-run {
display: flex;
height: 33px;
padding: 4px 10px;
justify-content: center;
align-items: center;
gap: 6px;
flex: 1 0 0;
border-radius: 2px;
background: var(--danger-background);
border: none;
}
.cancel-run:hover {
background-color: var(--bg-cherry-400) !important;
color: var(--l1-foreground) !important;
}
}

View File

@@ -329,14 +329,13 @@ describe('transformTransactionGroupsToResourcePermissions', () => {
it('returns all resources from RESOURCE_ORDER even with empty transaction groups', () => {
const result = transformTransactionGroupsToResourcePermissions([]);
expect(result).toHaveLength(10);
expect(result).toHaveLength(9);
expect(result.map((r) => r.resourceKind)).toStrictEqual([
'factor-api-key',
'license',
'logs',
'meter-metrics',
'metrics',
'quick-filter',
'role',
'serviceaccount',
'subscription',
@@ -421,14 +420,13 @@ describe('createEmptyRolePermissions', () => {
it('creates permissions for all resources in RESOURCE_ORDER', () => {
const result = createEmptyRolePermissions();
expect(result).toHaveLength(10);
expect(result).toHaveLength(9);
expect(result.map((r) => r.resourceKind)).toStrictEqual([
'factor-api-key',
'license',
'logs',
'meter-metrics',
'metrics',
'quick-filter',
'role',
'serviceaccount',
'subscription',

View File

@@ -5,7 +5,6 @@ import {
FileKey,
Gauge,
Key,
ListFilter,
Logs,
Receipt,
Shield,
@@ -78,14 +77,6 @@ export const RESOURCE_PANELS: Record<AuthZResource, ResourcePanelConfig> = {
selectorPlaceholder: 'Type * to cover the workspace subscription',
docsAnchor: 'subscription',
},
'quick-filter': {
label: 'Quick Filters',
description: 'Quick filters shown in the logs, traces, and other explorers.',
icon: ListFilter,
selectorPlaceholder:
'Type quick filter ID, separate multiple with comma or space',
docsAnchor: 'quick-filter',
},
logs: {
label: 'Logs',
description: 'Log data collected across the workspace.',

View File

@@ -13,11 +13,6 @@ export default {
type: 'metaresource',
allowedVerbs: ['create', 'delete', 'list', 'read', 'update'],
},
{
kind: 'quick-filter',
type: 'metaresource',
allowedVerbs: ['list', 'read', 'update'],
},
{
kind: 'subscription',
type: 'metaresource',

View File

@@ -1,16 +0,0 @@
import { buildPermission } from '../utils';
export const QuickFilterReadPermission = buildPermission(
'read',
'quick-filter:*',
);
export const QuickFilterUpdatePermission = buildPermission(
'update',
'quick-filter:*',
);
// Editing quick filters needs read as well as update
export const QuickFilterManagePermissions = [
QuickFilterReadPermission,
QuickFilterUpdatePermission,
];

1
go.mod
View File

@@ -57,7 +57,6 @@ require (
github.com/segmentio/analytics-go/v3 v3.2.1
github.com/sethvargo/go-password v0.2.0
github.com/smartystreets/goconvey v1.8.1
github.com/soheilhy/cmux v0.1.5
github.com/spf13/cobra v1.10.2
github.com/stretchr/testify v1.11.1
github.com/swaggest/jsonschema-go v0.3.78

3
go.sum
View File

@@ -1057,8 +1057,6 @@ github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1
github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA=
github.com/smartystreets/goconvey v1.8.1 h1:qGjIddxOk4grTu9JPOU31tVfq3cNdBlNa5sSznIX1xY=
github.com/smartystreets/goconvey v1.8.1/go.mod h1:+/u4qLyY6x1jReYOp7GOM2FSt8aP9CzCZL03bI28W60=
github.com/soheilhy/cmux v0.1.5 h1:jjzc5WVemNEDTLwv9tlmemhC73tI08BNOIGwBOo10Js=
github.com/soheilhy/cmux v0.1.5/go.mod h1:T7TcVDs9LWfQgPlPsdngu6I6QIoyIFZDDC6sNE1GqG0=
github.com/sourcegraph/conc v0.3.0 h1:OQTbbt6P72L20UqAkXXuLOj79LfEanQ+YQFNpLA9ySo=
github.com/sourcegraph/conc v0.3.0/go.mod h1:Sdozi7LEKbFPqYX2/J+iBAM6HpqSLTASQIKqDmF7Mt0=
github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
@@ -1493,7 +1491,6 @@ golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81R
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201031054903-ff519b6c9102/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201202161906-c7110b5ffcbb/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201209123823-ac852fbbde11/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20201224014010-6772e930b67b/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210119194325-5f4716e94777/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=

View File

@@ -1,10 +1,14 @@
package apiserver
import (
"github.com/SigNoz/signoz/pkg/factory"
"github.com/gorilla/mux"
)
type APIServer interface {
// APIServer is a long running service serving the SigNoz API.
factory.ServiceWithHealthy
// Returns the mux router for the API server. Primarily used for collecting OpenAPI operations.
Router() *mux.Router

View File

@@ -3,11 +3,14 @@ package apiserver
import (
"time"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
)
// Config holds the configuration for config.
type Config struct {
// Address is the TCP address the API server listens on, in the form "host:port".
Address string `mapstructure:"address"`
Timeout Timeout `mapstructure:"timeout"`
Logging Logging `mapstructure:"logging"`
}
@@ -17,6 +20,8 @@ type Timeout struct {
Default time.Duration `mapstructure:"default"`
// The maximum allowed context timeout
Max time.Duration `mapstructure:"max"`
// RequestRead bounds reading an entire request, including the body, at the socket level.
RequestRead time.Duration `mapstructure:"request_read"`
// The list of routes that are excluded from the timeout
ExcludedRoutes []string `mapstructure:"excluded_routes"`
}
@@ -32,9 +37,11 @@ func NewConfigFactory() factory.ConfigFactory {
func newConfig() factory.Config {
return &Config{
Address: "0.0.0.0:8080",
Timeout: Timeout{
Default: 60 * time.Second,
Max: 600 * time.Second,
Default: 60 * time.Second,
Max: 600 * time.Second,
RequestRead: 60 * time.Second,
ExcludedRoutes: []string{
"/api/v1/logs/tail",
"/api/v3/logs/livetail",
@@ -52,5 +59,9 @@ func newConfig() factory.Config {
}
func (c Config) Validate() error {
if c.Address == "" {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "apiserver.address is required")
}
return nil
}

View File

@@ -13,6 +13,7 @@ import (
)
func TestNewWithEnvProvider(t *testing.T) {
t.Setenv("SIGNOZ_APISERVER_ADDRESS", "0.0.0.0:9090")
t.Setenv("SIGNOZ_APISERVER_TIMEOUT_DEFAULT", "70s")
t.Setenv("SIGNOZ_APISERVER_TIMEOUT_MAX", "700s")
t.Setenv("SIGNOZ_APISERVER_TIMEOUT_EXCLUDED__ROUTES", "/excluded1,/excluded2")
@@ -38,9 +39,11 @@ func TestNewWithEnvProvider(t *testing.T) {
require.NoError(t, err)
expected := &Config{
Address: "0.0.0.0:9090",
Timeout: Timeout{
Default: 70 * time.Second,
Max: 700 * time.Second,
Default: 70 * time.Second,
Max: 700 * time.Second,
RequestRead: 60 * time.Second,
ExcludedRoutes: []string{
"/excluded1",
"/excluded2",

View File

@@ -2,9 +2,11 @@ package signozapiserver
import (
"context"
"net/http"
"github.com/SigNoz/signoz/pkg/alertmanager"
"github.com/SigNoz/signoz/pkg/apiserver"
"github.com/SigNoz/signoz/pkg/auditor"
"github.com/SigNoz/signoz/pkg/authz"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
@@ -12,6 +14,8 @@ import (
"github.com/SigNoz/signoz/pkg/global"
"github.com/SigNoz/signoz/pkg/http/handler"
"github.com/SigNoz/signoz/pkg/http/middleware"
httpserver "github.com/SigNoz/signoz/pkg/http/server"
"github.com/SigNoz/signoz/pkg/identn"
"github.com/SigNoz/signoz/pkg/licensing"
"github.com/SigNoz/signoz/pkg/modules/aiobservability"
"github.com/SigNoz/signoz/pkg/modules/authdomain"
@@ -37,18 +41,22 @@ import (
"github.com/SigNoz/signoz/pkg/prometheus"
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/ruler"
"github.com/SigNoz/signoz/pkg/sharder"
"github.com/SigNoz/signoz/pkg/statsreporter"
"github.com/SigNoz/signoz/pkg/subscription"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/web"
"github.com/SigNoz/signoz/pkg/zeus"
"github.com/gorilla/mux"
)
type provider struct {
config apiserver.Config
settings factory.ScopedProviderSettings
globalConfig global.Config
web web.Web
router *mux.Router
httpServer *httpserver.Server
healthyC chan struct{}
authzMiddleware *middleware.AuthZ
authzService authz.AuthZ
orgHandler organization.Handler
@@ -132,6 +140,11 @@ func NewFactory(
rulerHandler ruler.Handler,
statsHandler statsreporter.Handler,
savedViewHandler savedview.Handler,
globalConfig global.Config,
identNResolver identn.IdentNResolver,
sharder sharder.Sharder,
auditor auditor.Auditor,
web web.Web,
quickFilterModule quickfilter.Module,
quickFilterHandler quickfilter.Handler,
) factory.ProviderFactory[apiserver.APIServer, apiserver.Config] {
@@ -179,6 +192,11 @@ func NewFactory(
rulerHandler,
statsHandler,
savedViewHandler,
globalConfig,
identNResolver,
sharder,
auditor,
web,
quickFilterModule,
quickFilterHandler,
)
@@ -228,6 +246,11 @@ func newProvider(
rulerHandler ruler.Handler,
statsHandler statsreporter.Handler,
savedViewHandler savedview.Handler,
globalConfig global.Config,
identNResolver identn.IdentNResolver,
sharder sharder.Sharder,
auditor auditor.Auditor,
web web.Web,
quickFilterModule quickfilter.Module,
quickFilterHandler quickfilter.Handler,
) (apiserver.APIServer, error) {
@@ -235,9 +258,10 @@ func newProvider(
router := mux.NewRouter().UseEncodedPath()
provider := &provider{
config: config,
settings: settings,
globalConfig: globalConfig,
web: web,
router: router,
healthyC: make(chan struct{}),
orgHandler: orgHandler,
userHandler: userHandler,
authzService: authzService,
@@ -282,13 +306,68 @@ func newProvider(
provider.authzMiddleware = middleware.NewAuthZ(settings.Logger(), orgGetter, authzService)
router.Use(middleware.NewRecovery(settings.Logger()).Wrap)
router.Use(middleware.NewOtel("apiserver", providerSettings.MeterProvider, providerSettings.TracerProvider).Wrap)
router.Use(middleware.NewIdentN(identNResolver, sharder, settings.Logger()).Wrap)
router.Use(middleware.NewTimeout(settings.Logger(),
config.Timeout.ExcludedRoutes,
config.Timeout.Default,
config.Timeout.Max,
).Wrap)
router.Use(middleware.NewResource(settings.Logger()).Wrap)
router.Use(middleware.NewAudit(settings.Logger(), config.Logging.ExcludedRoutes, auditor).Wrap)
router.Use(middleware.NewComment().Wrap)
if err := provider.AddToRouter(router); err != nil {
return nil, err
}
httpHandler := middleware.NewCors().Wrap(router)
httpHandler = middleware.NewCompress().Wrap(httpHandler)
routePrefix := globalConfig.ExternalPath()
if routePrefix != "" {
prefixed := http.StripPrefix(routePrefix, httpHandler)
httpHandler = http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
switch req.URL.Path {
case "/api/v1/health", "/api/v2/healthz", "/api/v2/readyz", "/api/v2/livez":
router.ServeHTTP(w, req)
return
}
prefixed.ServeHTTP(w, req)
})
}
httpServer, err := httpserver.New(settings.Logger(), httpserver.Config{Address: config.Address, ReadTimeout: config.Timeout.RequestRead}, httpHandler)
if err != nil {
return nil, err
}
provider.httpServer = httpServer
return provider, nil
}
func (provider *provider) Start(ctx context.Context) error {
// Mount the web routes last so the catch-all prefix does not shadow API
// routes registered on the router after construction.
if err := provider.web.AddToRouter(provider.router); err != nil {
return err
}
close(provider.healthyC)
return provider.httpServer.Start(ctx)
}
func (provider *provider) Stop(ctx context.Context) error {
return provider.httpServer.Stop(ctx)
}
func (provider *provider) Healthy() <-chan struct{} {
return provider.healthyC
}
func (provider *provider) Router() *mux.Router {
return provider.router
}

View File

@@ -78,6 +78,40 @@ func NewRegistry(ctx context.Context, logger *slog.Logger, services ...NamedServ
}, nil
}
// Add registers additional services into the registry. It must be called before Start.
func (registry *Registry) Add(ctx context.Context, services ...NamedService) error {
added := make([]*serviceWithState, 0, len(services))
for _, s := range services {
if _, ok := registry.servicesByName[s.Name()]; ok {
return errors.Newf(errors.TypeInvalidInput, ErrCodeInvalidRegistry, "cannot add service, duplicate service name %q", s.Name())
}
added = append(added, newServiceWithState(s))
}
for _, ss := range added {
registry.services = append(registry.services, ss)
registry.servicesByName[ss.service.Name()] = ss
}
for _, ss := range added {
for _, dep := range ss.service.DependsOn() {
if dep == ss.service.Name() {
registry.logger.ErrorContext(ctx, "ignoring self-dependency", slog.Any("service", ss.service.Name()))
continue
}
if _, ok := registry.servicesByName[dep]; !ok {
registry.logger.ErrorContext(ctx, "ignoring unknown dependency", slog.Any("service", ss.service.Name()), slog.Any("dependency", dep))
continue
}
ss.dependsOn = append(ss.dependsOn, dep)
}
}
return detectCyclicDeps(registry.services)
}
func (registry *Registry) Start(ctx context.Context) {
for _, ss := range registry.services {
go func(ss *serviceWithState) {

View File

@@ -342,3 +342,61 @@ func TestDependsOnCycleReturnsError(t *testing.T) {
assert.Error(t, err)
assert.Contains(t, err.Error(), "dependency cycles detected")
}
func TestRegistryAdd(t *testing.T) {
s1 := newTestService(t)
s2 := newTestService(t)
registry, err := NewRegistry(context.Background(), slog.New(slog.DiscardHandler), NewNamedService(MustNewName("s1"), s1))
require.NoError(t, err)
require.NoError(t, registry.Add(context.Background(), NewNamedService(MustNewName("s2"), s2)))
ctx := context.Background()
registry.Start(ctx)
require.NoError(t, registry.AwaitHealthy(ctx))
byState := registry.ServicesByState()
assert.Len(t, byState[StateRunning], 2)
assert.True(t, registry.IsHealthy())
assert.NoError(t, registry.Stop(ctx))
}
func TestRegistryAddDuplicateReturnsError(t *testing.T) {
s1 := newTestService(t)
registry, err := NewRegistry(context.Background(), slog.New(slog.DiscardHandler), NewNamedService(MustNewName("s1"), s1))
require.NoError(t, err)
err = registry.Add(context.Background(), NewNamedService(MustNewName("s1"), newTestService(t)))
assert.Error(t, err)
assert.Contains(t, err.Error(), "duplicate service name")
}
func TestRegistryAddWithDependency(t *testing.T) {
s1 := newHealthyTestService(t)
s2 := newTestService(t)
registry, err := NewRegistry(context.Background(), slog.New(slog.DiscardHandler), NewNamedService(MustNewName("s1"), s1))
require.NoError(t, err)
// s2 depends on the already registered s1.
require.NoError(t, registry.Add(context.Background(), NewNamedService(MustNewName("s2"), s2, MustNewName("s1"))))
ctx := context.Background()
registry.Start(ctx)
// s2 stays in STARTING until s1 is healthy.
require.Eventually(t, func() bool {
byState := registry.ServicesByState()
return len(byState[StateStarting]) == 2
}, time.Second, time.Millisecond)
close(s1.healthyC)
require.NoError(t, registry.AwaitHealthy(ctx))
assert.True(t, registry.IsHealthy())
assert.NoError(t, registry.Stop(ctx))
}

View File

@@ -0,0 +1,17 @@
package middleware
import (
"net/http"
gorillahandlers "github.com/gorilla/handlers"
)
type Compress struct{}
func NewCompress() *Compress {
return &Compress{}
}
func (middleware *Compress) Wrap(next http.Handler) http.Handler {
return gorillahandlers.CompressHandler(next)
}

View File

@@ -0,0 +1,25 @@
package middleware
import (
"net/http"
"github.com/rs/cors"
)
type Cors struct {
cors *cors.Cors
}
func NewCors() *Cors {
return &Cors{
cors: cors.New(cors.Options{
AllowedOrigins: []string{"*"},
AllowedMethods: []string{"GET", "DELETE", "POST", "PUT", "PATCH", "OPTIONS"},
AllowedHeaders: []string{"Accept", "Authorization", "Content-Type", "cache-control"},
}),
}
}
func (middleware *Cors) Wrap(next http.Handler) http.Handler {
return middleware.cors.Handler(next)
}

View File

@@ -0,0 +1,43 @@
package middleware
import (
"net/http"
"slices"
"github.com/gorilla/mux"
"go.opentelemetry.io/contrib/instrumentation/github.com/gorilla/mux/otelmux"
"go.opentelemetry.io/otel/metric"
"go.opentelemetry.io/otel/propagation"
"go.opentelemetry.io/otel/trace"
)
// defaultExcludedRoutes are the health endpoints kept out of tracing/metrics to
// avoid drowning telemetry in probe traffic.
var defaultExcludedRoutes = []string{
"/api/v1/health",
"/api/v2/healthz",
"/api/v2/readyz",
"/api/v2/livez",
}
type Otel struct {
wrap mux.MiddlewareFunc
}
func NewOtel(service string, meterProvider metric.MeterProvider, tracerProvider trace.TracerProvider) *Otel {
return &Otel{
wrap: otelmux.Middleware(
service,
otelmux.WithMeterProvider(meterProvider),
otelmux.WithTracerProvider(tracerProvider),
otelmux.WithPropagators(propagation.NewCompositeTextMapPropagator(propagation.Baggage{}, propagation.TraceContext{})),
otelmux.WithFilter(func(r *http.Request) bool {
return !slices.Contains(defaultExcludedRoutes, r.URL.Path)
}),
),
}
}
func (middleware *Otel) Wrap(next http.Handler) http.Handler {
return middleware.wrap(next)
}

View File

@@ -1,9 +1,18 @@
package server
import "time"
// Config holds the configuration for http.
type Config struct {
//Address specifies the TCP address for the server to listen on, in the form "host:port".
// If empty, ":http" (port 80) is used. The service names are defined in RFC 6335 and assigned by IANA.
// See net.Dial for details of the address format.
Address string `mapstructure:"address"`
// ReadTimeout bounds reading an entire request, including the body. Zero means no timeout.
ReadTimeout time.Duration `mapstructure:"read_timeout"`
// WriteTimeout bounds writing the response. Zero means no timeout, required for
// streaming endpoints that hold the connection open.
WriteTimeout time.Duration `mapstructure:"write_timeout"`
}

View File

@@ -31,8 +31,8 @@ func New(logger *slog.Logger, cfg Config, handler http.Handler) (*Server, error)
srv := &http.Server{
Addr: cfg.Address,
Handler: handler,
ReadTimeout: 10 * time.Second,
WriteTimeout: 10 * time.Second,
ReadTimeout: cfg.ReadTimeout,
WriteTimeout: cfg.WriteTimeout,
MaxHeaderBytes: 1 << 20,
}

View File

@@ -82,7 +82,6 @@ func (handler *handler) GetFieldsValues(rw http.ResponseWriter, req *http.Reques
values := &telemetrytypes.TelemetryFieldValues{
StringValues: allValues.StringValues,
BoolValues: allValues.BoolValues,
NumberValues: allValues.NumberValues,
RelatedValues: relatedValues,
}

View File

@@ -6,6 +6,7 @@ import (
"net/http"
nethttppprof "net/http/pprof"
runtimepprof "runtime/pprof"
"time"
"github.com/SigNoz/signoz/pkg/factory"
httpserver "github.com/SigNoz/signoz/pkg/http/server"
@@ -23,7 +24,7 @@ func NewFactory() factory.ProviderFactory[pprof.PProf, pprof.Config] {
func New(_ context.Context, settings factory.ProviderSettings, config pprof.Config) (pprof.PProf, error) {
server, err := httpserver.New(
settings.Logger.With(slog.String("pkg", "github.com/SigNoz/signoz/pkg/pprof/httppprof")),
httpserver.Config{Address: config.Address},
httpserver.Config{Address: config.Address, ReadTimeout: 10 * time.Second, WriteTimeout: 10 * time.Second},
newHandler(),
)
if err != nil {

View File

@@ -2,19 +2,9 @@ package app
import (
"context"
"fmt"
"net"
"net/http"
"slices"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/queryparser"
"github.com/gorilla/handlers"
"github.com/rs/cors"
"github.com/soheilhy/cmux"
"github.com/SigNoz/signoz/pkg/http/middleware"
"github.com/SigNoz/signoz/pkg/query-service/agentConf"
"github.com/SigNoz/signoz/pkg/query-service/app/clickhouseReader"
@@ -23,31 +13,15 @@ import (
"github.com/SigNoz/signoz/pkg/query-service/app/opamp"
opAmpModel "github.com/SigNoz/signoz/pkg/query-service/app/opamp/model"
"github.com/SigNoz/signoz/pkg/signoz"
"github.com/SigNoz/signoz/pkg/web"
"log/slog"
"go.opentelemetry.io/contrib/instrumentation/github.com/gorilla/mux/otelmux"
"go.opentelemetry.io/otel/propagation"
"github.com/SigNoz/signoz/pkg/query-service/constants"
"github.com/SigNoz/signoz/pkg/query-service/healthcheck"
"github.com/SigNoz/signoz/pkg/query-service/utils"
)
// Server runs HTTP, Mux and a grpc server
// Server runs auxiliary servers (opamp) alongside the signoz apiserver
type Server struct {
config signoz.Config
signoz *signoz.SigNoz
// public http router
httpConn net.Listener
httpServer *http.Server
httpHostPort string
opampServer *opamp.Server
unavailableChannel chan healthcheck.Status
}
// NewServer creates and initializes Server
@@ -90,20 +64,20 @@ func NewServer(config signoz.Config, signoz *signoz.SigNoz) (*Server, error) {
return nil, err
}
s := &Server{
config: config,
signoz: signoz,
httpHostPort: constants.HTTPHostPort,
unavailableChannel: make(chan healthcheck.Status),
}
// Register the legacy query-service routes on the apiserver router. The
// apiserver owns the HTTP server and applies the middleware chain at serve
// time, so these routes get the same treatment as the apiserver routes.
r := signoz.APIServer.Router()
am := middleware.NewAuthZ(signoz.Instrumentation.Logger(), signoz.Modules.OrgGetter, signoz.Authz)
httpServer, err := s.createPublicServer(apiHandler, signoz.Web)
if err != nil {
return nil, err
}
s.httpServer = httpServer
apiHandler.RegisterRoutes(r, am)
apiHandler.RegisterLogsRoutes(r, am)
apiHandler.RegisterIntegrationRoutes(r, am)
apiHandler.RegisterQueryRangeV3Routes(r, am)
apiHandler.RegisterQueryRangeV4Routes(r, am)
apiHandler.RegisterMessagingQueuesRoutes(r, am)
apiHandler.RegisterThirdPartyApiRoutes(r, am)
apiHandler.RegisterTraceFunnelsRoutes(r, am)
opAmpModel.Init(signoz.SQLStore, signoz.Instrumentation.Logger(), signoz.Modules.OrgGetter)
@@ -121,6 +95,8 @@ func NewServer(config signoz.Config, signoz *signoz.SigNoz) (*Server, error) {
return nil, err
}
s := &Server{}
s.opampServer = opamp.InitializeServer(
&opAmpModel.AllAgents,
agentConfMgr,
@@ -130,146 +106,18 @@ func NewServer(config signoz.Config, signoz *signoz.SigNoz) (*Server, error) {
return s, nil
}
// HealthCheckStatus returns health check status channel a client can subscribe to
func (s Server) HealthCheckStatus() chan healthcheck.Status {
return s.unavailableChannel
}
func (s *Server) createPublicServer(api *APIHandler, web web.Web) (*http.Server, error) {
r := NewRouter()
r.Use(middleware.NewRecovery(s.signoz.Instrumentation.Logger()).Wrap)
r.Use(otelmux.Middleware(
"apiserver",
otelmux.WithMeterProvider(s.signoz.Instrumentation.MeterProvider()),
otelmux.WithTracerProvider(s.signoz.Instrumentation.TracerProvider()),
otelmux.WithPropagators(propagation.NewCompositeTextMapPropagator(propagation.Baggage{}, propagation.TraceContext{})),
otelmux.WithFilter(func(r *http.Request) bool {
return !slices.Contains([]string{"/api/v1/health"}, r.URL.Path)
}),
))
r.Use(middleware.NewIdentN(s.signoz.IdentNResolver, s.signoz.Sharder, s.signoz.Instrumentation.Logger()).Wrap)
r.Use(middleware.NewTimeout(s.signoz.Instrumentation.Logger(),
s.config.APIServer.Timeout.ExcludedRoutes,
s.config.APIServer.Timeout.Default,
s.config.APIServer.Timeout.Max,
).Wrap)
r.Use(middleware.NewResource(s.signoz.Instrumentation.Logger()).Wrap)
r.Use(middleware.NewAudit(s.signoz.Instrumentation.Logger(), s.config.APIServer.Logging.ExcludedRoutes, s.signoz.Auditor).Wrap)
r.Use(middleware.NewComment().Wrap)
am := middleware.NewAuthZ(s.signoz.Instrumentation.Logger(), s.signoz.Modules.OrgGetter, s.signoz.Authz)
api.RegisterRoutes(r, am)
api.RegisterLogsRoutes(r, am)
api.RegisterIntegrationRoutes(r, am)
api.RegisterQueryRangeV3Routes(r, am)
api.RegisterQueryRangeV4Routes(r, am)
api.RegisterMessagingQueuesRoutes(r, am)
api.RegisterThirdPartyApiRoutes(r, am)
api.RegisterTraceFunnelsRoutes(r, am)
err := s.signoz.APIServer.AddToRouter(r)
if err != nil {
return nil, err
}
c := cors.New(cors.Options{
AllowedOrigins: []string{"*"},
AllowedMethods: []string{"GET", "DELETE", "POST", "PUT", "PATCH", "OPTIONS"},
AllowedHeaders: []string{"Accept", "Authorization", "Content-Type", "cache-control"},
})
handler := c.Handler(r)
handler = handlers.CompressHandler(handler)
err = web.AddToRouter(r)
if err != nil {
return nil, err
}
routePrefix := s.config.Global.ExternalPath()
if routePrefix != "" {
prefixed := http.StripPrefix(routePrefix, handler)
handler = http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
switch req.URL.Path {
case "/api/v1/health", "/api/v2/healthz", "/api/v2/readyz", "/api/v2/livez":
r.ServeHTTP(w, req)
return
}
prefixed.ServeHTTP(w, req)
})
}
return &http.Server{
Handler: handler,
}, nil
}
// initListeners initialises listeners of the server
func (s *Server) initListeners() error {
// listen on public port
var err error
publicHostPort := s.httpHostPort
if publicHostPort == "" {
return fmt.Errorf("constants.HTTPHostPort is required")
}
s.httpConn, err = net.Listen("tcp", publicHostPort)
if err != nil {
return err
}
slog.Info(fmt.Sprintf("Query server started listening on %s...", s.httpHostPort))
return nil
}
// Start listening on http and private http port concurrently
// Start starts the opamp websocket server. The HTTP API server is started by
// the signoz registry.
func (s *Server) Start(ctx context.Context) error {
err := s.initListeners()
if err != nil {
slog.Info("Starting OpAmp Websocket server", "addr", constants.OpAmpWsEndpoint)
if err := s.opampServer.Start(constants.OpAmpWsEndpoint); err != nil {
return err
}
var httpPort int
if port, err := utils.GetPort(s.httpConn.Addr()); err == nil {
httpPort = port
}
go func() {
slog.Info("Starting HTTP server", "port", httpPort, "addr", s.httpHostPort)
switch err := s.httpServer.Serve(s.httpConn); err {
case nil, http.ErrServerClosed, cmux.ErrListenerClosed:
// normal exit, nothing to do
default:
slog.Error("Could not start HTTP server", errors.Attr(err))
}
s.unavailableChannel <- healthcheck.Unavailable
}()
go func() {
slog.Info("Starting OpAmp Websocket server", "addr", constants.OpAmpWsEndpoint)
err := s.opampServer.Start(constants.OpAmpWsEndpoint)
if err != nil {
slog.Error("opamp ws server failed to start", errors.Attr(err))
s.unavailableChannel <- healthcheck.Unavailable
}
}()
return nil
}
func (s *Server) Stop(ctx context.Context) error {
if s.httpServer != nil {
if err := s.httpServer.Shutdown(context.Background()); err != nil {
return err
}
}
s.opampServer.Stop()
return nil

View File

@@ -10,11 +10,7 @@ import (
"github.com/SigNoz/signoz/pkg/valuer"
)
const (
HTTPHostPort = "0.0.0.0:8080" // Address to serve http (query service)
PrivateHostPort = "0.0.0.0:8085" // Address to server internal services like alert manager
OpAmpWsEndpoint = "0.0.0.0:4320" // address for opamp websocket
)
const OpAmpWsEndpoint = "0.0.0.0:4320" // address for opamp websocket
const MaxAllowedPointsInTimeSeries = 300

View File

@@ -1,12 +0,0 @@
package healthcheck
const (
// Unavailable indicates the service is not able to handle requests
Unavailable Status = iota
// Ready indicates the service is ready to handle requests
Ready
// Broken indicates that the healthcheck itself is broken, not serving HTTP
Broken
)
type Status int

View File

@@ -10,12 +10,14 @@ import (
"github.com/SigNoz/signoz/pkg/alertmanager"
"github.com/SigNoz/signoz/pkg/apiserver"
"github.com/SigNoz/signoz/pkg/apiserver/signozapiserver"
"github.com/SigNoz/signoz/pkg/auditor"
"github.com/SigNoz/signoz/pkg/authz"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/gateway"
"github.com/SigNoz/signoz/pkg/global"
"github.com/SigNoz/signoz/pkg/http/handler"
"github.com/SigNoz/signoz/pkg/identn"
"github.com/SigNoz/signoz/pkg/instrumentation"
"github.com/SigNoz/signoz/pkg/licensing"
"github.com/SigNoz/signoz/pkg/modules/aiobservability"
@@ -42,9 +44,11 @@ import (
"github.com/SigNoz/signoz/pkg/prometheus"
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/ruler"
"github.com/SigNoz/signoz/pkg/sharder"
"github.com/SigNoz/signoz/pkg/statsreporter"
"github.com/SigNoz/signoz/pkg/subscription"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/web"
"github.com/SigNoz/signoz/pkg/zeus"
"github.com/swaggest/jsonschema-go"
"github.com/swaggest/openapi-go"
@@ -101,6 +105,11 @@ func NewOpenAPI(ctx context.Context, instrumentation instrumentation.Instrumenta
struct{ ruler.Handler }{},
struct{ statsreporter.Handler }{},
struct{ savedview.Handler }{},
global.Config{},
struct{ identn.IdentNResolver }{},
struct{ sharder.Sharder }{},
struct{ auditor.Auditor }{},
struct{ web.Web }{},
struct{ quickfilter.Module }{},
struct{ quickfilter.Handler }{},
).New(ctx, instrumentation.ToProviderSettings(), apiserver.Config{})

View File

@@ -253,7 +253,6 @@ func NewSQLMigrationProviderFactories(
sqlmigration.NewAddQuickFilterTuplesFactory(sqlstore),
sqlmigration.NewAddIngestionTuplesFactory(sqlstore),
sqlmigration.NewAddSubscriptionTuplesFactory(sqlstore),
sqlmigration.NewNormalizeQuickFilterFieldsFactory(sqlstore),
)
}
@@ -319,7 +318,7 @@ func NewQuerierProviderFactories(telemetryStore telemetrystore.TelemetryStore, p
)
}
func NewAPIServerProviderFactories(orgGetter organization.Getter, authz authz.AuthZ, modules Modules, handlers Handlers, globalConfig global.Config, gatewayService gateway.Gateway) factory.NamedMap[factory.ProviderFactory[apiserver.APIServer, apiserver.Config]] {
func NewAPIServerProviderFactories(orgGetter organization.Getter, authz authz.AuthZ, modules Modules, handlers Handlers, globalConfig global.Config, gatewayService gateway.Gateway, identNResolver identn.IdentNResolver, sharder sharder.Sharder, auditor auditor.Auditor, web web.Web) factory.NamedMap[factory.ProviderFactory[apiserver.APIServer, apiserver.Config]] {
return factory.MustNewNamedMap(
signozapiserver.NewFactory(
orgGetter,
@@ -361,6 +360,11 @@ func NewAPIServerProviderFactories(orgGetter organization.Getter, authz authz.Au
handlers.RulerHandler,
handlers.StatsHandler,
handlers.SavedView,
globalConfig,
identNResolver,
sharder,
auditor,
web,
modules.QuickFilter,
handlers.QuickFilter,
),

View File

@@ -102,6 +102,10 @@ func TestNewProviderFactories(t *testing.T) {
Handlers{},
global.Config{},
nil,
nil,
nil,
nil,
nil,
)
})
}

View File

@@ -635,13 +635,20 @@ func New(
ctx,
providerSettings,
config.APIServer,
NewAPIServerProviderFactories(orgGetter, authz, modules, handlers, config.Global, gateway),
NewAPIServerProviderFactories(orgGetter, authz, modules, handlers, config.Global, gateway, identNResolver, sharder, auditor, web),
"signoz",
)
if err != nil {
return nil, err
}
// Register the API server with the registry so its lifecycle is managed
// alongside the other services and it shows up in the health endpoint.
err = registry.Add(ctx, factory.NewNamedService(factory.MustNewName("apiserver"), apiserverInstance))
if err != nil {
return nil, err
}
return &SigNoz{
Registry: registry,
Analytics: analytics,

View File

@@ -1,161 +0,0 @@
package sqlmigration
import (
"context"
"encoding/json"
"log/slog"
"github.com/uptrace/bun"
"github.com/uptrace/bun/migrate"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/sqlstore"
)
type quickFilterSourceRow struct {
bun.BaseModel `bun:"table:quick_filter"`
ID string `bun:"id,pk"`
Source string `bun:"source"`
Filter string `bun:"filter"`
}
type quickFilterStaticField struct {
name string
fieldContext string
fieldDataType string
}
// quickFilterSpanFields are the span-level fields the fields API serves with
// the span context, keyed by every name a stored filter may carry for them.
var quickFilterSpanFields = func() map[string]quickFilterStaticField {
fields := map[string]quickFilterStaticField{}
for name, dataType := range map[string]string{
"trace_id": "string", "span_id": "string", "trace_state": "string", "parent_span_id": "string",
"flags": "number", "name": "string", "kind": "number", "kind_string": "string",
"duration_nano": "number", "status_code": "number", "status_message": "string", "status_code_string": "string",
"response_status_code": "string", "external_http_url": "string", "http_url": "string",
"external_http_method": "string", "http_method": "string", "http_host": "string",
"db_name": "string", "db_operation": "string", "has_error": "bool", "is_remote": "string",
} {
fields[name] = quickFilterStaticField{name: name, fieldContext: "span", fieldDataType: dataType}
}
for deprecated, current := range map[string]string{
"responseStatusCode": "response_status_code", "externalHttpUrl": "external_http_url", "httpUrl": "http_url",
"externalHttpMethod": "external_http_method", "httpMethod": "http_method", "httpHost": "http_host",
"dbName": "db_name", "dbOperation": "db_operation", "hasError": "has_error", "isRemote": "is_remote",
} {
fields[deprecated] = fields[current]
}
return fields
}()
// quickFilterLogFields are the log-level fields the fields API serves with
// the log context.
var quickFilterLogFields = map[string]quickFilterStaticField{
"body": {name: "body", fieldContext: "log", fieldDataType: "string"},
"severity_text": {name: "severity_text", fieldContext: "log", fieldDataType: "string"},
"severity_number": {name: "severity_number", fieldContext: "log", fieldDataType: "number"},
"trace_id": {name: "trace_id", fieldContext: "log", fieldDataType: "string"},
"span_id": {name: "span_id", fieldContext: "log", fieldDataType: "string"},
"trace_flags": {name: "trace_flags", fieldContext: "log", fieldDataType: "number"},
}
type normalizeQuickFilterFields struct {
settings factory.ProviderSettings
}
func NewNormalizeQuickFilterFieldsFactory(sqlstore sqlstore.SQLStore) factory.ProviderFactory[SQLMigration, Config] {
return factory.NewProviderFactory(factory.MustNewName("normalize_quick_filter_fields"), func(ctx context.Context, ps factory.ProviderSettings, c Config) (SQLMigration, error) {
return &normalizeQuickFilterFields{settings: ps}, nil
})
}
func (migration *normalizeQuickFilterFields) Register(migrations *migrate.Migrations) error {
return migrations.Register(migration.Up, migration.Down)
}
func (migration *normalizeQuickFilterFields) Up(ctx context.Context, db *bun.DB) error {
tx, err := db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer func() { _ = tx.Rollback() }()
var rows []*quickFilterSourceRow
if err := tx.NewSelect().Model(&rows).Scan(ctx); err != nil {
return err
}
var migrated, skipped int
for _, row := range rows {
normalized, changed, ok := normalizeQuickFilterEntries(row.Source, row.Filter)
if !ok {
migration.settings.Logger.WarnContext(ctx, "quick filter could not be parsed, leaving it untouched", slog.String("quick_filter_id", row.ID), slog.String("raw_filter", row.Filter))
skipped++
continue
}
if !changed {
continue
}
migrated++
if _, err := tx.NewUpdate().Model((*quickFilterSourceRow)(nil)).Set("filter = ?", normalized).Where("id = ?", row.ID).Exec(ctx); err != nil {
return err
}
}
migration.settings.Logger.InfoContext(ctx, "normalized quick filter static fields", slog.Int("total", len(rows)), slog.Int("migrated", migrated), slog.Int("skipped", skipped))
return tx.Commit()
}
func (migration *normalizeQuickFilterFields) Down(context.Context, *bun.DB) error {
return nil
}
// normalizeQuickFilterEntries rewrites the static fields of a stored filter
// list to the name, context and data type the fields API serves them with:
// span fields for the trace-based sources, log fields for logs, whatever
// context the legacy seeds gave them. Other keys are left as they are;
// ok=false means unparseable.
func normalizeQuickFilterEntries(source string, filter string) (normalized string, changed bool, ok bool) {
var staticFields map[string]quickFilterStaticField
switch source {
case "traces", "api_monitoring", "exceptions", "ai_observability":
staticFields = quickFilterSpanFields
case "logs":
staticFields = quickFilterLogFields
default:
return "", false, true
}
var entries []telemetryFieldKeyOutput
if err := json.Unmarshal([]byte(filter), &entries); err != nil {
return "", false, false
}
for i, entry := range entries {
field, static := staticFields[entry.Name]
if !static {
continue
}
if entry.Name == field.name && entry.FieldContext == field.fieldContext && entry.FieldDataType == field.fieldDataType {
continue
}
entries[i].Name = field.name
entries[i].FieldContext = field.fieldContext
entries[i].FieldDataType = field.fieldDataType
changed = true
}
if !changed {
return "", false, true
}
normalizedJSON, err := marshalUnescaped(entries)
if err != nil {
return "", false, false
}
return string(normalizedJSON), true, true
}

View File

@@ -1,61 +0,0 @@
package telemetrymetadata
import (
"strconv"
"strings"
"github.com/SigNoz/signoz/pkg/telemetryschema/tracestelemetryschema"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
)
// boolFieldValues is the suggestion set for a bool field, optionally narrowed
// by the search text.
func boolFieldValues(searchText string) *telemetrytypes.TelemetryFieldValues {
values := &telemetrytypes.TelemetryFieldValues{}
needle := strings.ToLower(searchText)
for _, v := range []bool{true, false} {
if needle == "" || strings.Contains(strconv.FormatBool(v), needle) {
values.BoolValues = append(values.BoolValues, v)
}
}
return values
}
// spanSearchScopeFieldValues is the suggestion set for a search-scope selector
// (isRoot, isEntryPoint), which only filters with true. ok is false for any
// other name.
func spanSearchScopeFieldValues(name, searchText string) (*telemetrytypes.TelemetryFieldValues, bool) {
for scopeName := range tracestelemetryschema.SpanSearchScopeFields {
if !strings.EqualFold(scopeName, name) {
continue
}
values := &telemetrytypes.TelemetryFieldValues{}
if needle := strings.ToLower(searchText); needle == "" || strings.Contains("true", needle) {
values.BoolValues = []bool{true}
}
return values, true
}
return nil, false
}
// isKnownBoolField is true when the caller asked for the bool data type, or
// when the name is one of the signal's static bool fields and the requested
// context does not rule that static field out.
func isKnownBoolField(selector *telemetrytypes.FieldValueSelector, staticFields ...map[string]telemetrytypes.TelemetryFieldKey) bool {
if selector.FieldDataType == telemetrytypes.FieldDataTypeBool {
return true
}
if selector.FieldDataType != telemetrytypes.FieldDataTypeUnspecified {
return false
}
for _, fields := range staticFields {
field, ok := fields[selector.Name]
if !ok || field.FieldDataType != telemetrytypes.FieldDataTypeBool {
continue
}
if selector.FieldContext == telemetrytypes.FieldContextUnspecified || selector.FieldContext == field.FieldContext {
return true
}
}
return false
}

View File

@@ -187,6 +187,8 @@ func (t *telemetryMetaStore) getTracesKeys(ctx context.Context, fieldKeySelector
).From(t.tracesDBName + "." + t.spanAttributesKeysTblName)
var limit int
searchTexts := []string{}
conds := []string{}
for _, fieldKeySelector := range fieldKeySelectors {
@@ -206,12 +208,14 @@ func (t *telemetryMetaStore) getTracesKeys(ctx context.Context, fieldKeySelector
fieldKeyConds = append(fieldKeyConds, sb.ILike("tagKey", "%"+escapeForLike(fieldKeySelector.Name)+"%"))
}
searchTexts = append(searchTexts, fieldKeySelector.Name)
// now look at the field context
// we don't write most of intrinsic fields to keys table
// for this reason we don't want to apply tagType if the field context
// is not attribute, resource attribute or scope
switch fieldKeySelector.FieldContext {
case telemetrytypes.FieldContextAttribute, telemetrytypes.FieldContextResource, telemetrytypes.FieldContextScope:
// is not attribute or resource attribute
if fieldKeySelector.FieldContext != telemetrytypes.FieldContextUnspecified &&
(fieldKeySelector.FieldContext == telemetrytypes.FieldContextAttribute ||
fieldKeySelector.FieldContext == telemetrytypes.FieldContextResource) {
fieldKeyConds = append(fieldKeyConds, sb.E("tagType", fieldKeySelector.FieldContext.TagType()))
}
@@ -284,20 +288,41 @@ func (t *telemetryMetaStore) getTracesKeys(ctx context.Context, fieldKeySelector
// hit the limit? (only counting DB results)
complete := rowCount <= limit
// Add the matching static fields: the span scope selectors, the intrinsic
// columns and the calculated columns. These don't count towards the limit
staticFields := maps.Values(tracestelemetryschema.SpanSearchScopeFields)
staticFields = append(staticFields, maps.Values(tracestelemetryschema.IntrinsicFields)...)
staticFields = append(staticFields, maps.Values(tracestelemetryschema.CalculatedFields)...)
staticKeys := []string{"isRoot", "isEntryPoint"}
staticKeys = append(staticKeys, maps.Keys(tracestelemetryschema.IntrinsicFields)...)
staticKeys = append(staticKeys, maps.Keys(tracestelemetryschema.CalculatedFields)...)
for _, field := range staticFields {
if !staticFieldMatchesAny(field, fieldKeySelectors) {
continue
// Add matching intrinsic and matching calculated fields
// These don't count towards the limit
for _, key := range staticKeys {
found := false
for _, v := range searchTexts {
if v == "" || strings.Contains(key, v) {
found = true
break
}
}
if _, added := mapOfKeys[field.Name+";"+field.FieldContext.StringValue()+";"+field.FieldDataType.StringValue()]; added {
continue
if found {
if field, exists := tracestelemetryschema.IntrinsicFields[key]; exists {
if _, added := mapOfKeys[field.Name+";"+field.FieldContext.StringValue()+";"+field.FieldDataType.StringValue()]; !added {
keys = append(keys, &field)
}
continue
}
if field, exists := tracestelemetryschema.CalculatedFields[key]; exists {
if _, added := mapOfKeys[field.Name+";"+field.FieldContext.StringValue()+";"+field.FieldDataType.StringValue()]; !added {
keys = append(keys, &field)
}
continue
}
keys = append(keys, &telemetrytypes.TelemetryFieldKey{
Name: key,
FieldContext: telemetrytypes.FieldContextSpan,
Signal: telemetrytypes.SignalTraces,
})
}
keys = append(keys, &field)
}
if err = t.updateColumnEvolutionMetadataForKeys(ctx, keys); err != nil {
@@ -517,6 +542,12 @@ func (t *telemetryMetaStore) getLogsKeys(ctx context.Context, orgID valuer.UUID,
allArgs = append(allArgs, args...)
}
if len(queries) == 0 {
// No matching contexts, return empty result
return []*telemetrytypes.TelemetryFieldKey{}, true, nil
}
// Combine queries with UNION ALL
var limit int
for _, fieldKeySelector := range fieldKeySelectors {
limit += fieldKeySelector.Limit
@@ -525,15 +556,7 @@ func (t *telemetryMetaStore) getLogsKeys(ctx context.Context, orgID valuer.UUID,
limit = 1000
}
keys := []*telemetrytypes.TelemetryFieldKey{}
parentTypes := make(map[string][]telemetrytypes.FieldDataType)
rowCount := 0
// the log and scope contexts have no keys table; they are served by the
// static fields appended below
if len(queries) > 0 {
// Combine queries with UNION ALL
mainQuery := fmt.Sprintf(`
mainQuery := fmt.Sprintf(`
SELECT tag_key, tag_type, tag_data_type, max(priority) as priority
FROM (
%s
@@ -543,75 +566,103 @@ func (t *telemetryMetaStore) getLogsKeys(ctx context.Context, orgID valuer.UUID,
LIMIT %d
`, strings.Join(queries, " UNION ALL "), limit+1)
rows, err := t.telemetrystore.ClickhouseDB().Query(ctx, mainQuery, allArgs...)
rows, err := t.telemetrystore.ClickhouseDB().Query(ctx, mainQuery, allArgs...)
if err != nil {
return nil, false, errors.Wrap(err, errors.TypeInternal, errors.CodeInternal, ErrFailedToGetLogsKeys.Error())
}
defer rows.Close()
keys := []*telemetrytypes.TelemetryFieldKey{}
parentTypes := make(map[string][]telemetrytypes.FieldDataType)
rowCount := 0
searchTexts := []string{}
// Collect search texts for static field matching
for _, fieldKeySelector := range fieldKeySelectors {
searchTexts = append(searchTexts, fieldKeySelector.Name)
}
for rows.Next() {
rowCount++
// reached the limit, we know there are more results
if rowCount > limit {
break
}
var name string
var fieldContext telemetrytypes.FieldContext
var fieldDataType telemetrytypes.FieldDataType
var priority uint8
err = rows.Scan(&name, &fieldContext, &fieldDataType, &priority)
if err != nil {
return nil, false, errors.Wrap(err, errors.TypeInternal, errors.CodeInternal, ErrFailedToGetLogsKeys.Error())
}
defer rows.Close()
for rows.Next() {
rowCount++
// reached the limit, we know there are more results
if rowCount > limit {
break
}
var name string
var fieldContext telemetrytypes.FieldContext
var fieldDataType telemetrytypes.FieldDataType
var priority uint8
err = rows.Scan(&name, &fieldContext, &fieldDataType, &priority)
if err != nil {
return nil, false, errors.Wrap(err, errors.TypeInternal, errors.CodeInternal, ErrFailedToGetLogsKeys.Error())
}
// ArrayJSON/ArrayDynamic body rows for parent paths are needed by the JSON access plan
// builder (enrichJSONKeys). Always record them in parentTypes. Only skip adding to keys
// if the user did not also directly request this name — a field like "education" can be
// both a parent of "education[].name" and an explicitly queried field in its own right.
switch fieldDataType {
case telemetrytypes.FieldDataTypeArrayJSON, telemetrytypes.FieldDataTypeArrayDynamic:
if fieldContext == telemetrytypes.FieldContextBody && parentPaths[name] {
parentTypes[name] = append(parentTypes[name], fieldDataType)
if !mapOfRequestedSelectors[name] {
continue // skip; don't register the key.
}
// ArrayJSON/ArrayDynamic body rows for parent paths are needed by the JSON access plan
// builder (enrichJSONKeys). Always record them in parentTypes. Only skip adding to keys
// if the user did not also directly request this name — a field like "education" can be
// both a parent of "education[].name" and an explicitly queried field in its own right.
switch fieldDataType {
case telemetrytypes.FieldDataTypeArrayJSON, telemetrytypes.FieldDataTypeArrayDynamic:
if fieldContext == telemetrytypes.FieldContextBody && parentPaths[name] {
parentTypes[name] = append(parentTypes[name], fieldDataType)
if !mapOfRequestedSelectors[name] {
continue // skip; don't register the key.
}
}
}
key, ok := mapOfKeys[name+";"+fieldContext.StringValue()+";"+fieldDataType.StringValue()]
key, ok := mapOfKeys[name+";"+fieldContext.StringValue()+";"+fieldDataType.StringValue()]
// if there is no materialised column, create a key with the field context and data type
if !ok {
key = &telemetrytypes.TelemetryFieldKey{
Name: name,
Signal: telemetrytypes.SignalLogs,
FieldContext: fieldContext,
FieldDataType: fieldDataType,
}
// if there is no materialised column, create a key with the field context and data type
if !ok {
key = &telemetrytypes.TelemetryFieldKey{
Name: name,
Signal: telemetrytypes.SignalLogs,
FieldContext: fieldContext,
FieldDataType: fieldDataType,
}
keys = append(keys, key)
mapOfKeys[name+";"+fieldContext.StringValue()+";"+fieldDataType.StringValue()] = key
}
if rows.Err() != nil {
return nil, false, errors.Wrap(rows.Err(), errors.TypeInternal, errors.CodeInternal, ErrFailedToGetLogsKeys.Error())
}
keys = append(keys, key)
mapOfKeys[name+";"+fieldContext.StringValue()+";"+fieldDataType.StringValue()] = key
}
if rows.Err() != nil {
return nil, false, errors.Wrap(rows.Err(), errors.TypeInternal, errors.CodeInternal, ErrFailedToGetLogsKeys.Error())
}
// hit the limit? (only counting DB results)
complete := rowCount <= limit
// Add the matching intrinsic columns. These don't count towards the limit
for _, field := range maps.Values(logstelemetryschema.IntrinsicFields) {
if !staticFieldMatchesAny(field, fieldKeySelectors) {
continue
staticKeys := []string{}
staticKeys = append(staticKeys, maps.Keys(logstelemetryschema.IntrinsicFields)...)
// Add matching intrinsic and matching calculated fields
// These don't count towards the limit
for _, key := range staticKeys {
found := false
for _, v := range searchTexts {
if v == "" || strings.Contains(key, v) {
found = true
break
}
}
if _, added := mapOfKeys[field.Name+";"+field.FieldContext.StringValue()+";"+field.FieldDataType.StringValue()]; added {
continue
if found {
if field, exists := logstelemetryschema.IntrinsicFields[key]; exists {
if _, added := mapOfKeys[field.Name+";"+field.FieldContext.StringValue()+";"+field.FieldDataType.StringValue()]; !added {
keys = append(keys, &field)
}
continue
}
keys = append(keys, &telemetrytypes.TelemetryFieldKey{
Name: key,
FieldContext: telemetrytypes.FieldContextLog,
Signal: telemetrytypes.SignalLogs,
})
}
keys = append(keys, &field)
}
// enrich body keys with promoted paths, indexes, and JSON access plans
@@ -755,6 +806,10 @@ func (t *telemetryMetaStore) getAuditKeys(ctx context.Context, fieldKeySelectors
allArgs = append(allArgs, args...)
}
if len(queries) == 0 {
return []*telemetrytypes.TelemetryFieldKey{}, true, nil
}
var limit int
for _, fieldKeySelector := range fieldKeySelectors {
limit += fieldKeySelector.Limit
@@ -763,13 +818,7 @@ func (t *telemetryMetaStore) getAuditKeys(ctx context.Context, fieldKeySelectors
limit = 1000
}
keys := []*telemetrytypes.TelemetryFieldKey{}
rowCount := 0
// the log and scope contexts have no keys table; they are served by the
// static fields appended below
if len(queries) > 0 {
mainQuery := fmt.Sprintf(`
mainQuery := fmt.Sprintf(`
SELECT tag_key, tag_type, tag_data_type, max(priority) as priority
FROM (
%s
@@ -779,57 +828,73 @@ func (t *telemetryMetaStore) getAuditKeys(ctx context.Context, fieldKeySelectors
LIMIT %d
`, strings.Join(queries, " UNION ALL "), limit+1)
rows, err := t.telemetrystore.ClickhouseDB().Query(ctx, mainQuery, allArgs...)
rows, err := t.telemetrystore.ClickhouseDB().Query(ctx, mainQuery, allArgs...)
if err != nil {
return nil, false, errors.Wrap(err, errors.TypeInternal, errors.CodeInternal, ErrFailedToGetAuditKeys.Error())
}
defer rows.Close()
keys := []*telemetrytypes.TelemetryFieldKey{}
rowCount := 0
searchTexts := []string{}
for _, fieldKeySelector := range fieldKeySelectors {
searchTexts = append(searchTexts, fieldKeySelector.Name)
}
for rows.Next() {
rowCount++
if rowCount > limit {
break
}
var name string
var fieldContext telemetrytypes.FieldContext
var fieldDataType telemetrytypes.FieldDataType
var priority uint8
err = rows.Scan(&name, &fieldContext, &fieldDataType, &priority)
if err != nil {
return nil, false, errors.Wrap(err, errors.TypeInternal, errors.CodeInternal, ErrFailedToGetAuditKeys.Error())
}
defer rows.Close()
key, ok := mapOfKeys[name+";"+fieldContext.StringValue()+";"+fieldDataType.StringValue()]
for rows.Next() {
rowCount++
if rowCount > limit {
break
if !ok {
key = &telemetrytypes.TelemetryFieldKey{
Name: name,
Signal: telemetrytypes.SignalLogs,
FieldContext: fieldContext,
FieldDataType: fieldDataType,
}
var name string
var fieldContext telemetrytypes.FieldContext
var fieldDataType telemetrytypes.FieldDataType
var priority uint8
err = rows.Scan(&name, &fieldContext, &fieldDataType, &priority)
if err != nil {
return nil, false, errors.Wrap(err, errors.TypeInternal, errors.CodeInternal, ErrFailedToGetAuditKeys.Error())
}
key, ok := mapOfKeys[name+";"+fieldContext.StringValue()+";"+fieldDataType.StringValue()]
if !ok {
key = &telemetrytypes.TelemetryFieldKey{
Name: name,
Signal: telemetrytypes.SignalLogs,
FieldContext: fieldContext,
FieldDataType: fieldDataType,
}
}
keys = append(keys, key)
mapOfKeys[name+";"+fieldContext.StringValue()+";"+fieldDataType.StringValue()] = key
}
if rows.Err() != nil {
return nil, false, errors.Wrap(rows.Err(), errors.TypeInternal, errors.CodeInternal, ErrFailedToGetAuditKeys.Error())
}
keys = append(keys, key)
mapOfKeys[name+";"+fieldContext.StringValue()+";"+fieldDataType.StringValue()] = key
}
if rows.Err() != nil {
return nil, false, errors.Wrap(rows.Err(), errors.TypeInternal, errors.CodeInternal, ErrFailedToGetAuditKeys.Error())
}
complete := rowCount <= limit
// Add the matching intrinsic audit fields (same as logs intrinsics: body, severity_text, etc.)
for _, field := range maps.Values(audittelemetryschema.IntrinsicFields) {
if !staticFieldMatchesAny(field, fieldKeySelectors) {
continue
// Add intrinsic audit fields (same as logs intrinsics: body, severity_text, etc.)
staticKeys := maps.Keys(audittelemetryschema.IntrinsicFields)
for _, key := range staticKeys {
found := false
for _, v := range searchTexts {
if v == "" || strings.Contains(key, v) {
found = true
break
}
}
if _, added := mapOfKeys[field.Name+";"+field.FieldContext.StringValue()+";"+field.FieldDataType.StringValue()]; added {
continue
if found {
if field, exists := audittelemetryschema.IntrinsicFields[key]; exists {
if _, added := mapOfKeys[field.Name+";"+field.FieldContext.StringValue()+";"+field.FieldDataType.StringValue()]; !added {
keys = append(keys, &field)
}
}
}
keys = append(keys, &field)
}
return keys, complete, nil
@@ -1026,12 +1091,9 @@ func (t *telemetryMetaStore) getMeterSourceMetricKeys(ctx context.Context, field
if err != nil {
return nil, false, errors.Wrap(err, errors.TypeInternal, errors.CodeInternal, ErrFailedToGetMeterKeys.Error())
}
// meter labels are stored as strings in the labels JSON and have no
// attribute context, so only the data type is known
keys = append(keys, &telemetrytypes.TelemetryFieldKey{
Name: name,
Signal: telemetrytypes.SignalMetrics,
FieldDataType: telemetrytypes.FieldDataTypeString,
Name: name,
Signal: telemetrytypes.SignalMetrics,
})
}
@@ -1444,13 +1506,88 @@ func (t *telemetryMetaStore) getSpanFieldValues(ctx context.Context, fieldValueS
instrumentationtypes.CodeNamespace: "metadata",
instrumentationtypes.CodeFunctionName: "getSpanFieldValues",
})
if values, ok := spanSearchScopeFieldValues(fieldValueSelector.Name, fieldValueSelector.Value); ok {
return values, true, nil
// build the query to get the keys from the spans that match the field selection criteria
limit := fieldValueSelector.Limit
if limit == 0 {
limit = 50
}
knownBool := isKnownBoolField(fieldValueSelector, tracestelemetryschema.IntrinsicFields, tracestelemetryschema.CalculatedFields)
// unix_milli is the hour of the span start
return t.getTagTableValues(ctx, t.tracesDBName+"."+t.tracesFieldsTblName, fieldValueSelector, knownBool)
sb := sqlbuilder.Select("DISTINCT string_value, number_value").From(t.tracesDBName + "." + t.tracesFieldsTblName)
if fieldValueSelector.Name != "" {
sb.Where(sb.E("tag_key", fieldValueSelector.Name))
}
// now look at the field context
if fieldValueSelector.FieldContext != telemetrytypes.FieldContextUnspecified {
sb.Where(sb.E("tag_type", fieldValueSelector.FieldContext.TagType()))
}
// now look at the field data type
if fieldValueSelector.FieldDataType != telemetrytypes.FieldDataTypeUnspecified {
sb.Where(sb.E("tag_data_type", fieldValueSelector.FieldDataType.TagDataType()))
}
if fieldValueSelector.Value != "" {
switch fieldValueSelector.FieldDataType {
case telemetrytypes.FieldDataTypeString:
sb.Where(sb.ILike("string_value", "%"+escapeForLike(fieldValueSelector.Value)+"%"))
case telemetrytypes.FieldDataTypeNumber:
sb.Where(sb.IsNotNull("number_value"))
sb.Where(sb.ILike("toString(number_value)", "%"+escapeForLike(fieldValueSelector.Value)+"%"))
case telemetrytypes.FieldDataTypeUnspecified:
// or b/w string and number
sb.Where(sb.Or(
sb.ILike("string_value", "%"+escapeForLike(fieldValueSelector.Value)+"%"),
sb.ILike("toString(number_value)", "%"+escapeForLike(fieldValueSelector.Value)+"%"),
))
}
}
// query one extra to check if we hit the limit
sb.Limit(limit + 1)
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
rows, err := t.telemetrystore.ClickhouseDB().Query(ctx, query, args...)
if err != nil {
return nil, false, errors.Wrap(err, errors.TypeInternal, errors.CodeInternal, ErrFailedToGetLogsKeys.Error())
}
defer rows.Close()
values := &telemetrytypes.TelemetryFieldValues{}
seen := make(map[string]bool)
rowCount := 0
totalCount := 0 // Track total unique values
for rows.Next() {
rowCount++
var stringValue string
var numberValue float64
if err := rows.Scan(&stringValue, &numberValue); err != nil {
return nil, false, errors.Wrap(err, errors.TypeInternal, errors.CodeInternal, ErrFailedToGetLogsKeys.Error())
}
// Only add values if we haven't hit the limit yet
if totalCount < limit {
if _, ok := seen[stringValue]; !ok && stringValue != "" {
values.StringValues = append(values.StringValues, stringValue)
seen[stringValue] = true
totalCount++
}
if _, ok := seen[fmt.Sprintf("%f", numberValue)]; !ok && numberValue != 0 && totalCount < limit {
values.NumberValues = append(values.NumberValues, numberValue)
seen[fmt.Sprintf("%f", numberValue)] = true
totalCount++
}
}
}
// hit the limit?
complete := rowCount <= limit
return values, complete, nil
}
func (t *telemetryMetaStore) getLogFieldValues(ctx context.Context, fieldValueSelector *telemetrytypes.FieldValueSelector) (*telemetrytypes.TelemetryFieldValues, bool, error) {
@@ -1459,77 +1596,17 @@ func (t *telemetryMetaStore) getLogFieldValues(ctx context.Context, fieldValueSe
instrumentationtypes.CodeNamespace: "metadata",
instrumentationtypes.CodeFunctionName: "getLogFieldValues",
})
knownBool := isKnownBoolField(fieldValueSelector, logstelemetryschema.IntrinsicFields)
// unix_milli is the hour the log was ingested, not the log's own timestamp
return t.getTagTableValues(ctx, t.logsDBName+"."+t.logsFieldsTblName, fieldValueSelector, knownBool)
}
// tagTableSinceDay restricts rows to the tag table's day partitions from the
// start's day on. Partitions are toDate(unix_milli / 1000) in the server's
// timezone, and a value's surviving row within a day carries whichever hour
// was inserted last, so the day is the finest safe unit.
func tagTableSinceDay(sb *sqlbuilder.SelectBuilder, startUnixMilli int64) {
if startUnixMilli != 0 {
sb.Where(fmt.Sprintf("toDate(unix_milli / 1000) >= toDate(%d)", startUnixMilli/1000))
}
}
// tagTableHasBoolRows reports whether the tag table holds a bool row for the
// key. Bool rows carry no value, so one row is enough to know the key takes
// the values true and false.
func (t *telemetryMetaStore) tagTableHasBoolRows(ctx context.Context, table string, selector *telemetrytypes.FieldValueSelector) (bool, error) {
sb := sqlbuilder.Select("1").From(table)
sb.Where(sb.E("tag_key", selector.Name))
sb.Where(sb.E("tag_data_type", telemetrytypes.FieldDataTypeBool.TagDataType()))
if selector.FieldContext != telemetrytypes.FieldContextUnspecified {
sb.Where(sb.E("tag_type", selector.FieldContext.TagType()))
}
tagTableSinceDay(sb, selector.StartUnixMilli)
sb.Limit(1)
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
rows, err := t.telemetrystore.ClickhouseDB().Query(ctx, query, args...)
if err != nil {
return false, errors.Wrap(err, errors.TypeInternal, errors.CodeInternal, ErrFailedToGetLogsKeys.Error())
}
defer rows.Close()
return rows.Next(), rows.Err()
}
// getTagTableValues returns the string and number values of the key from a
// tag table, and true and false when the key is a known bool field or the
// table holds bool rows for it. Bool rows do not count towards the limit.
func (t *telemetryMetaStore) getTagTableValues(ctx context.Context, table string, fieldValueSelector *telemetrytypes.FieldValueSelector, knownBool bool) (*telemetrytypes.TelemetryFieldValues, bool, error) {
// build the query to get the keys from the spans that match the field selection criteria
limit := fieldValueSelector.Limit
if limit == 0 {
limit = 50
}
values := &telemetrytypes.TelemetryFieldValues{}
if knownBool {
values.BoolValues = boolFieldValues(fieldValueSelector.Value).BoolValues
if fieldValueSelector.FieldDataType == telemetrytypes.FieldDataTypeBool {
return values, true, nil
}
} else if fieldValueSelector.FieldDataType == telemetrytypes.FieldDataTypeUnspecified {
hasBoolRows, err := t.tagTableHasBoolRows(ctx, table, fieldValueSelector)
if err != nil {
return nil, false, err
}
if hasBoolRows {
values.BoolValues = boolFieldValues(fieldValueSelector.Value).BoolValues
}
}
sb := sqlbuilder.Select("DISTINCT string_value, number_value").From(table)
sb := sqlbuilder.Select("DISTINCT string_value, number_value").From(t.logsDBName + "." + t.logsFieldsTblName)
if fieldValueSelector.Name != "" {
sb.Where(sb.E("tag_key", fieldValueSelector.Name))
}
sb.Where(sb.NE("tag_data_type", telemetrytypes.FieldDataTypeBool.TagDataType()))
tagTableSinceDay(sb, fieldValueSelector.StartUnixMilli)
if fieldValueSelector.FieldContext != telemetrytypes.FieldContextUnspecified {
sb.Where(sb.E("tag_type", fieldValueSelector.FieldContext.TagType()))
@@ -1566,6 +1643,7 @@ func (t *telemetryMetaStore) getTagTableValues(ctx context.Context, table string
}
defer rows.Close()
values := &telemetrytypes.TelemetryFieldValues{}
seen := make(map[string]bool)
rowCount := 0
totalCount := 0 // Track total unique values
@@ -2019,18 +2097,6 @@ func populateAllUnspecifiedValues(allUnspecifiedValues *telemetrytypes.Telemetry
}
}
for _, value := range values.BoolValues {
if totalCount >= limit {
complete = false
break
}
if _, ok := mapOfValues[value]; !ok {
mapOfValues[value] = true
allUnspecifiedValues.BoolValues = append(allUnspecifiedValues.BoolValues, value)
totalCount++
}
}
for _, value := range values.RelatedValues {
if totalCount >= limit {
complete = false
@@ -2401,10 +2467,6 @@ func (k *telemetryMetaStore) fetchEvolutionEntryFromClickHouse(ctx context.Conte
// updateColumnEvolutionMetadataForKeys updates the evolution field for keys.
func (k *telemetryMetaStore) updateColumnEvolutionMetadataForKeys(ctx context.Context, keysToUpdate []*telemetrytypes.TelemetryFieldKey) error {
// an empty selector list would run the evolution query without a filter
if len(keysToUpdate) == 0 {
return nil
}
var metadataKeySelectors []*telemetrytypes.EvolutionSelector
for _, keySelector := range keysToUpdate {

View File

@@ -1,53 +0,0 @@
package telemetrymetadata
import (
"strings"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
)
func staticFieldMatchesAny(field telemetrytypes.TelemetryFieldKey, selectors []*telemetrytypes.FieldKeySelector) bool {
for _, selector := range selectors {
if staticFieldMatches(field, selector) {
return true
}
}
return false
}
// staticFieldMatches mirrors the keys-table lookup for a static field: the
// requested context and data type, when given, must agree with the field's,
// and the name matches case-insensitively, as a substring for fuzzy selectors
// and as the whole name for exact ones.
func staticFieldMatches(field telemetrytypes.TelemetryFieldKey, selector *telemetrytypes.FieldKeySelector) bool {
if selector.FieldContext != telemetrytypes.FieldContextUnspecified && selector.FieldContext != field.FieldContext {
return false
}
if selector.FieldDataType != telemetrytypes.FieldDataTypeUnspecified && !sameDataTypeFamily(selector.FieldDataType, field.FieldDataType) {
return false
}
if selector.Name == "" {
return true
}
if selector.SelectorMatchType == telemetrytypes.FieldSelectorMatchTypeExact {
return strings.EqualFold(field.Name, selector.Name)
}
return strings.Contains(strings.ToLower(field.Name), strings.ToLower(selector.Name))
}
// sameDataTypeFamily treats the numeric types as one family: static fields
// declare "number" while callers may ask for int64 or float64.
func sameDataTypeFamily(requested, actual telemetrytypes.FieldDataType) bool {
if requested == actual {
return true
}
return isNumericDataType(requested) && isNumericDataType(actual)
}
func isNumericDataType(dataType telemetrytypes.FieldDataType) bool {
switch dataType {
case telemetrytypes.FieldDataTypeNumber, telemetrytypes.FieldDataTypeInt64, telemetrytypes.FieldDataTypeFloat64:
return true
}
return false
}

View File

@@ -392,24 +392,6 @@ var (
SpanSearchScopeRoot = "isroot"
SpanSearchScopeEntryPoint = "isentrypoint"
// SpanSearchScopeFields are the search-scope selectors (isRoot, isEntryPoint),
// not columns and unrelated to the instrumentation scope: they only filter
// with the value true.
SpanSearchScopeFields = map[string]telemetrytypes.TelemetryFieldKey{
"isRoot": {
Name: "isRoot",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextSpan,
FieldDataType: telemetrytypes.FieldDataTypeBool,
},
"isEntryPoint": {
Name: "isEntryPoint",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextSpan,
FieldDataType: telemetrytypes.FieldDataTypeBool,
},
}
// IntrinsicSpanFields lists the intrinsic span columns, in the order they
// should appear when a raw query expands its SelectFields.
IntrinsicSpanFields = []telemetrytypes.TelemetryFieldKey{

View File

@@ -173,18 +173,18 @@ func NewSourceFilterFromStorableQuickFilter(storableQuickFilter *StorableQuickFi
// NewDefaultQuickFilter generates default filters for all supported sources.
func NewDefaultQuickFilter(orgID valuer.UUID) ([]*StorableQuickFilter, error) {
tracesFilters := []telemetrytypes.TelemetryFieldKey{
{Name: "duration_nano", FieldContext: telemetrytypes.FieldContextSpan, FieldDataType: telemetrytypes.FieldDataTypeNumber},
{Name: "duration_nano", FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeNumber},
{Name: "deployment.environment", FieldContext: telemetrytypes.FieldContextResource, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "has_error", FieldContext: telemetrytypes.FieldContextSpan, FieldDataType: telemetrytypes.FieldDataTypeBool},
{Name: "hasError", FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeBool},
{Name: "service.name", FieldContext: telemetrytypes.FieldContextResource, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "name", FieldContext: telemetrytypes.FieldContextSpan, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "name", FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "rpc.method", FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "response_status_code", FieldContext: telemetrytypes.FieldContextSpan, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "http_host", FieldContext: telemetrytypes.FieldContextSpan, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "response_status_code", FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "http_host", FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "http.method", FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "http.route", FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "http_url", FieldContext: telemetrytypes.FieldContextSpan, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "trace_id", FieldContext: telemetrytypes.FieldContextSpan, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "http_url", FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "trace_id", FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeString},
}
logsFilters := []telemetrytypes.TelemetryFieldKey{

View File

@@ -1,235 +0,0 @@
from collections.abc import Callable
from datetime import UTC, datetime, timedelta
from http import HTTPStatus
import pytest
import requests
from fixtures import types
from fixtures.auth import USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD
from fixtures.logs import Logs
from fixtures.traces import Traces
@pytest.mark.parametrize(
"signal,field_context,present,absent",
[
pytest.param("logs", "log", {"severity_text": "log", "body": "log", "trace_id": "log"}, ["code.file", "scope_name"], id="log_context_lists_log_intrinsics"),
pytest.param("logs", "scope", {"scope_name": "scope", "scope_version": "scope"}, ["severity_text", "body", "code.file"], id="scope_context_lists_scope_intrinsics_for_logs"),
pytest.param("logs", "attribute", {"code.file": "attribute"}, ["body", "scope_name"], id="attribute_context_excludes_log_intrinsics"),
pytest.param("traces", "span", {"name": "span", "has_error": "span", "isRoot": "span", "http.method": "attribute"}, ["scope.name"], id="span_context_lists_span_intrinsics_and_attributes"),
pytest.param("traces", "scope", {"scope.name": "scope", "scope.version": "scope"}, ["name", "has_error", "isRoot", "http.method", "host.name"], id="scope_context_lists_scope_intrinsics_for_traces"),
pytest.param("traces", "resource", {"host.name": "resource"}, ["name", "has_error", "isRoot", "http.method"], id="resource_context_excludes_span_intrinsics"),
pytest.param("traces", "attribute", {"http.method": "attribute"}, ["name", "has_error", "isRoot", "host.name"], id="attribute_context_excludes_span_intrinsics"),
],
)
def test_fields_keys_by_context(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_logs: Callable[[list[Logs]], None],
insert_traces: Callable[[list[Traces]], None],
signal: str,
field_context: str,
present: dict[str, str],
absent: list[str],
) -> None:
"""
Setup:
Insert a log with a code.file attribute and a span with an http.method attribute and a host.name resource.
Tests:
1. Keys for a context list that context's intrinsic columns and the stored keys the context maps to,
each with its context; intrinsics of other contexts are not listed. The span context also keeps
listing attributes because `span.<attribute>` resolves attributes in queries.
"""
now = datetime.now(tz=UTC)
insert_logs([Logs(timestamp=now, attributes={"code.file": "/opt/integration.go"}, body="a log line")])
insert_traces([Traces(timestamp=now, resources={"host.name": "linux-001"}, attributes={"http.method": "GET"})])
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v1/fields/keys"),
timeout=2,
headers={"authorization": f"Bearer {token}"},
params={"signal": signal, "fieldContext": field_context},
)
assert response.status_code == HTTPStatus.OK
keys = response.json()["data"]["keys"]
listed = {name: [key["fieldContext"] for key in keys.get(name, [])] for name in present}
assert listed == {name: [context] for name, context in present.items()}, f"keys for the {field_context} context"
assert [name for name in absent if name in keys] == [], f"keys that do not belong to the {field_context} context"
@pytest.mark.parametrize(
"signal,field_context,field_data_type,present,absent",
[
pytest.param("traces", "span", "float64", ["duration_nano", "status_code"], ["name", "has_error"], id="float64_matches_number_span_intrinsics"),
pytest.param("traces", "span", "int64", ["duration_nano", "status_code"], ["name", "has_error"], id="int64_matches_number_span_intrinsics"),
pytest.param("traces", "span", "bool", ["has_error", "isRoot", "isEntryPoint"], ["name", "duration_nano"], id="bool_matches_bool_span_intrinsics"),
pytest.param("traces", "span", "string", ["name", "http_method"], ["duration_nano", "has_error"], id="string_matches_string_span_intrinsics"),
pytest.param("logs", "log", "number", ["severity_number", "trace_flags"], ["severity_text", "body"], id="number_matches_number_log_intrinsics"),
],
)
def test_fields_keys_by_data_type(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
signal: str,
field_context: str,
field_data_type: str,
present: list[str],
absent: list[str],
) -> None:
"""
Tests:
1. A data type filter keeps the intrinsic columns of that type; number, int64 and float64 are one family.
"""
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v1/fields/keys"),
timeout=2,
headers={"authorization": f"Bearer {token}"},
params={"signal": signal, "fieldContext": field_context, "fieldDataType": field_data_type},
)
assert response.status_code == HTTPStatus.OK
keys = response.json()["data"]["keys"]
assert [name for name in present if name not in keys] == [], f"intrinsics of type {field_data_type}"
assert [name for name in absent if name in keys] == [], f"intrinsics not of type {field_data_type}"
@pytest.mark.parametrize(
"signal,search_text,present",
[
pytest.param("logs", "SEVERITY", ["severity_text", "severity_number"], id="upper_case_search_logs"),
pytest.param("traces", "HTTP_", ["http_method", "http_host", "http_url"], id="upper_case_search_traces"),
pytest.param("traces", "Duration", ["duration_nano"], id="mixed_case_search_traces"),
pytest.param("traces", "span.HAS_ERR", ["has_error"], id="context_prefix_with_upper_case_search"),
],
)
def test_fields_keys_search_matches_intrinsics_case_insensitively(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
signal: str,
search_text: str,
present: list[str],
) -> None:
"""
Tests:
1. The search text matches intrinsic columns case-insensitively, as it does for stored keys,
with or without a context prefix.
"""
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v1/fields/keys"),
timeout=2,
headers={"authorization": f"Bearer {token}"},
params={"signal": signal, "searchText": search_text},
)
assert response.status_code == HTTPStatus.OK
keys = response.json()["data"]["keys"]
assert [name for name in present if name not in keys] == [], f"intrinsics matching {search_text!r}"
@pytest.mark.parametrize(
"signal,params,expected",
[
pytest.param("traces", {"name": "has_error"}, [True, False], id="calculated_bool_span_field"),
pytest.param("traces", {"name": "has_error", "fieldContext": "span"}, [True, False], id="calculated_bool_span_field_with_context"),
pytest.param("traces", {"name": "has_error", "searchText": "tr"}, [True], id="search_text_narrows_bool_values"),
pytest.param("traces", {"name": "isRoot"}, [True], id="span_scope_field_is_true_only"),
pytest.param("logs", {"name": "retry"}, [True, False], id="bool_attribute_from_tag_rows"),
pytest.param("logs", {"name": "retry", "fieldContext": "attribute"}, [True, False], id="bool_attribute_with_context"),
pytest.param("logs", {"name": "retry", "searchText": "tr"}, [True], id="search_text_narrows_stored_bool_values"),
pytest.param("logs", {"name": "never_seen", "fieldDataType": "bool"}, [True, False], id="declared_bool_type_needs_no_rows"),
],
)
def test_fields_values_bool_fields(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_logs: Callable[[list[Logs]], None],
signal: str,
params: dict[str, str],
expected: list[bool],
) -> None:
"""
Setup:
Insert a log with a bool attribute.
Tests:
1. Values for a bool field are true and false (narrowed by the search text): for the calculated span
field, for a stored bool attribute whose tag rows carry no value, and for a key the caller
declares bool.
2. A span scope selector (isRoot) only takes true.
"""
insert_logs([Logs(timestamp=datetime.now(tz=UTC), attributes={"retry": True}, body="retrying")])
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v1/fields/values"),
timeout=2,
headers={"authorization": f"Bearer {token}"},
params={"signal": signal, **params},
)
assert response.status_code == HTTPStatus.OK
assert response.json()["data"]["values"]["boolValues"] == expected
assert response.json()["data"]["complete"] is True
def test_fields_values_start_excludes_span_values_not_seen_since_the_day(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""
Setup:
Insert a span three days old and a span now, with different service names.
Tests:
1. Values with startUnixMilli an hour ago contain only the service seen today: the start is
floored to the day, the tag table's deduplication unit.
2. Values without a start contain both services.
Logs are not covered: the logs collector stamps tag rows with the ingestion hour, not the
log's timestamp, and the fixture writes the log's timestamp.
"""
signal = "traces"
now = datetime.now(tz=UTC)
insert_traces(
[
Traces(timestamp=now - timedelta(days=3), resources={"service.name": "archived-service"}),
Traces(timestamp=now, resources={"service.name": "live-service"}),
]
)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v1/fields/values"),
timeout=2,
headers={"authorization": f"Bearer {token}"},
params={
"signal": signal,
"name": "service.name",
"startUnixMilli": int((now - timedelta(hours=1)).timestamp() * 1000),
},
)
assert response.status_code == HTTPStatus.OK
assert response.json()["data"]["values"]["stringValues"] == ["live-service"], "values last seen before the start must be dropped"
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v1/fields/values"),
timeout=2,
headers={"authorization": f"Bearer {token}"},
params={"signal": signal, "name": "service.name"},
)
assert response.status_code == HTTPStatus.OK
assert set(response.json()["data"]["values"]["stringValues"]) == {"archived-service", "live-service"}

View File

@@ -71,7 +71,7 @@ def test_v1_get_serves_legacy_shape(
assert response.status_code == HTTPStatus.OK, response.text
filters = response.json()["data"]["filters"]
assert filters[0]["key"] == "duration_nano"
assert filters[0]["type"] == "", "span fields have no v3 attribute type"
assert filters[0]["type"] == "tag"
assert filters[0]["dataType"] == "float64"
assert all("name" not in legacy_filter for legacy_filter in filters)
@@ -274,36 +274,3 @@ def test_update_quick_filters_rejects_invalid_input(
timeout=2,
)
assert response.status_code == HTTPStatus.BAD_REQUEST, response.text
def test_default_traces_filters_are_served_as_the_fields_api_serves_them(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
):
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v2/quick_filters/traces"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.OK, response.text
filters = {field_key["name"]: field_key for field_key in response.json()["data"]["filters"]}
assert "hasError" not in filters
assert (filters["has_error"]["fieldContext"], filters["has_error"]["fieldDataType"]) == ("span", "bool")
assert (filters["name"]["fieldContext"], filters["name"]["fieldDataType"]) == ("span", "string")
assert (filters["duration_nano"]["fieldContext"], filters["duration_nano"]["fieldDataType"]) == ("span", "number")
assert (filters["http.route"]["fieldContext"], filters["http.route"]["fieldDataType"]) == ("attribute", "string")
for name in ("has_error", "name"):
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v1/fields/keys"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
params={"signal": "traces", "searchText": name, "fieldContext": filters[name]["fieldContext"]},
)
assert response.status_code == HTTPStatus.OK, response.text
served = response.json()["data"]["keys"][name]
assert (filters[name]["fieldContext"], filters[name]["fieldDataType"]) in [(key["fieldContext"], key["fieldDataType"]) for key in served]