Compare commits

..

5 Commits

Author SHA1 Message Date
Naman Verma
67878a01b2 feat: add ai builder query plugin kind 2026-09-10 12:08:52 +05:30
Nikhil Soni
c9ae10b1c0 feat(apiserver): move apiserver to registry and make it configurable (#12493)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
#### Description

- Make server port configurable so multiple instances can be started for
agentic development and testing.
- Add `make go-stop` to make it easier to restart server by agents. It
does a graceful stop to allow the Prometheus metrics exporter port to
shutdown otherwise that port remain occupied.
- Add make target for generating the OpenAPI specs.
- Documents the above in `docs/contributing/development.md` under "How
do I run more than one instance?", including `make go-stop` in the basic
backend flow.

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-09 17:12:05 +00:00
Pandey
84a802edba docs(security): use GitHub private vulnerability reporting (#12820)
#### Description

- Switches the vulnerability reporting channel in `SECURITY.md` to
GitHub's private vulnerability reporting (Security → Report a
vulnerability), which is already enabled on this repo.
- Keeps `security@signoz.io` as a fallback for reporters who can't use
GitHub.
- Adds a short how-to and sets expectations for what happens after a
report (private advisory, coordinated fix, credit + CVE).
- Adds a `CODEOWNERS` entry so `SECURITY.md` is owned by @therealpandey.

#### Additional Information

- Docs / repo config only; no code changes.
2026-09-09 13:52:53 +00:00
Pandey
f78bd492d8 fix(tracefunnel): require view access on trace funnel analytics endpoints (#12817)
#### Description

- The twelve `/api/v1/trace-funnels/analytics/*` route registrations
were missing an authorization wrapper. The six payload routes
(`/analytics/*`) were reachable without authentication; the six
`/{funnel_id}/analytics/*` routes ran without the role check the
middleware applies.
- Wrap all twelve in `am.ViewAccess`, matching the sibling trace-funnel
CRUD routes, so they require an authenticated viewer.

#### Additional Information

- Applies to both community and enterprise editions
(`RegisterTraceFunnelsRoutes` is shared by both servers).
- No change for authenticated viewers; anonymous and role-less callers
now get 401/403.
- Security advisory:
https://github.com/SigNoz/signoz/security/advisories/GHSA-v549-7j2x-qjm5.
2026-09-09 12:39:46 +00:00
Gaurav Tewari
99dcd79979 chore(llm-observability): fork shared traces explorer files (#12795)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
#### Description

- Copies, unmodified, the traces explorer files that the AI
observability explorer is about to diverge from. The point is the *next*
PR (#12796): with this baseline landed, that diff shows only the real
changes instead of ~1000 lines of brand-new files.
- No behaviour change. Nothing outside the fork imports the new copies
yet — the explorer's views are wired to them in #12796, alongside the
changes that need them.


Sources, all taken from this PR's merge base:

| Copied to `container/LLMObservability/` | Source |
| --- | --- |
| `ToolbarActions/LeftToolbarActions.tsx` |
`container/QueryBuilder/components/ToolbarActions/LeftToolbarActions.tsx`
|
| `ToolbarActions/ToolbarActions.styles.scss` |
`container/QueryBuilder/components/ToolbarActions/ToolbarActions.styles.scss`
|
| `Explorer/aiActions.ts` | `pages/TracesExplorer/aiActions.ts` |
| `Explorer/Controls/*` | `container/TracesExplorer/Controls/*` |
| `Explorer/TraceLoading/*` | `container/TracesExplorer/TraceLoading/*`
|
| `Explorer/TracesTable/*` | `container/TracesExplorer/TracesTable/*` |
| `Explorer/ListView/utils.tsx` |
`container/TracesExplorer/ListView/utils.tsx` |

#### Issues closed by this PR

#### Screenshots / Screen Recordings

#### Additional Information

---------

Co-authored-by: Gaurav Tewari <tewarig@users.noreply.github.com>
2026-09-09 09:40:13 +00:00
93 changed files with 1965 additions and 2587 deletions

7
.claude/opencode.json Normal file
View File

@@ -0,0 +1,7 @@
{
"$schema": "https://opencode.ai/config.json",
"lsp": true,
"experimental": {
"disable_paste_summary": true
}
}

View File

@@ -7,5 +7,6 @@ Applies to everything in the repo — code, config, workflows.
- **Rationale goes in prose, not source.** Why a version is pinned, why a job exists, how a subsystem fits together — that belongs in the README or the PR.
- **Never remove pre-existing comments** when editing code. The bar above applies to comments you write, not comments already there.
- **Never talk to the reviewer.** No comments about where a change came from, what was changed, or why the change is correct — that belongs in the PR description and is noise the moment it merges.
- **Less is more.** When writing something intended for human consumption, (comment, commit message, reply to prompt) use as few words as possible. Pick every word meticulously to reduce the volume to a strict minimum. Be down to the point. Less is more.
Language rules build on this one: [`go-comments`](go-comments.md), [`py-comments`](py-comments.md).

View File

@@ -0,0 +1,16 @@
---
paths:
- "**/*.go"
---
# Contribution guidelines
- When making Go changes, always ensure they follow the contributing guildelines in [`docs/contributing/go/`](../../docs/contributing/go/).
- Look for existing patterns in the codebase for any change before implementing the changes.
- If any API contract is modified, generate the OpenAPI specs with `make gen-openapi-specs`.
- Always keep the OpenAPI spec generated in a separate commit, so the whole commit can be dropped in case of conflicts during merge. Do not try to resolve conflict in generated files, instead just generate them again.
- Avoid breaking function calls unncessarily into multilines for couple of arguments.
- Try to keep most computational only logic in types package itself related to a domain type, use modules as the orchestraction layer cordinating different layers and all db queries in store layer. Check the serviceaccount modules for inspiration when confused.
- When defining types, keep the structure of file to have any constants and variables first, then exported types and exported methods and then finally the unexported types and methods.
- Never import types or other modules in migration files, duplicate the required type or method to keep migration free from changes.
- Always run the gofmt tool for formating beforing commiting any changes.

View File

@@ -2,6 +2,10 @@
- **Follow the template** (`.github/pull_request_template.md`): fill in its headings (Description / Issues closed by this PR / Screenshots / Additional Information). Don't add sections the template doesn't have.
- **Keep only the headings that apply.** Delete every heading that has nothing under it, along with its `<!--...-->` placeholder comment. The body must never contain an empty heading — if only Description applies, the body has exactly that one heading.
- **Keep the description concise and human-readable.** A few plain bullets saying what changed and why, for a reviewer skimming it — not a wall of text, not a restatement of the diff, not generated boilerplate.
- **Keep the description concise and human-readable.** A few non repetitive bullets saying what changed and why, for a reviewer skimming it — not a wall of text, not a restatement of the diff, not generated boilerplate and not the user agent conversation details.
- **Reference issues with `Closes #issue-number`** under "Issues closed by this PR" so they auto-close on merge. This goes in the PR description only — never in commit messages.
- **Breaking changes can be added in additional information section** if any.
- **AI assistance in commits may optionally be disclosed with an `Assisted-by:` trailer** naming the model (e.g. `Assisted-by: Claude Opus 4.5`) — do NOT use a `Co-authored-by:` trailer for this.
- **Keep the commit body short and human readable** focused on decision made if any. Commit body must not re-iterate the changes done, skip if title is sufficient in conveying the change.
- **Use convensional commit format** for commits and PR title.
- **Do not amend the commits once pushed.** Always create a new commit once changes are pushed to remote.

4
.github/CODEOWNERS vendored
View File

@@ -15,6 +15,10 @@
.github @therealpandey
go.mod @therealpandey
# Security
/SECURITY.md @therealpandey
# Scaffold Owners
/pkg/config/ @therealpandey

1
.gitignore vendored
View File

@@ -232,3 +232,4 @@ pyrightconfig.json
# dev
.dev/
.claude/worktrees/
.claude/settings.local.json

View File

@@ -81,10 +81,13 @@ devenv-clickhouse-clean: ## Clean all ClickHouse data from filesystem
##############################################################
# go commands
##############################################################
SIGNOZ_SQLSTORE_SQLITE_PATH ?= signoz.db
SIGNOZ_APISERVER_ADDRESS ?= 0.0.0.0:8080
.PHONY: go-run-enterprise
go-run-enterprise: ## Runs the enterprise go backend server
@SIGNOZ_INSTRUMENTATION_LOGS_LEVEL=debug \
SIGNOZ_SQLSTORE_SQLITE_PATH=signoz.db \
SIGNOZ_SQLSTORE_SQLITE_PATH=$(SIGNOZ_SQLSTORE_SQLITE_PATH) \
SIGNOZ_WEB_ENABLED=false \
SIGNOZ_TOKENIZER_JWT_SECRET=secret \
SIGNOZ_ALERTMANAGER_PROVIDER=signoz \
@@ -101,7 +104,7 @@ go-test: ## Runs go unit tests
.PHONY: go-run-community
go-run-community: ## Runs the community go backend server
@SIGNOZ_INSTRUMENTATION_LOGS_LEVEL=debug \
SIGNOZ_SQLSTORE_SQLITE_PATH=signoz.db \
SIGNOZ_SQLSTORE_SQLITE_PATH=$(SIGNOZ_SQLSTORE_SQLITE_PATH) \
SIGNOZ_WEB_ENABLED=false \
SIGNOZ_TOKENIZER_JWT_SECRET=secret \
SIGNOZ_ALERTMANAGER_PROVIDER=signoz \
@@ -111,6 +114,28 @@ go-run-community: ## Runs the community go backend server
go run -race \
$(GO_BUILD_CONTEXT_COMMUNITY)/*.go server
.PHONY: go-stop
go-stop: ## Stops the go backend server listening on SIGNOZ_APISERVER_ADDRESS, waiting for it to release every port it holds
@PORT=$(lastword $(subst :, ,$(SIGNOZ_APISERVER_ADDRESS))); \
PIDS=$$(lsof -ti tcp:$$PORT); \
if [ -z "$$PIDS" ]; then \
echo "No signoz server running on port $$PORT."; \
echo "If it's running on a different port, rerun as: make go-stop SIGNOZ_APISERVER_ADDRESS=host:port"; \
exit 0; \
fi; \
kill $$PIDS 2>/dev/null; \
for i in $$(seq 1 10); do \
alive=$$(for p in $$PIDS; do kill -0 $$p 2>/dev/null && echo $$p; done); \
[ -z "$$alive" ] && break; \
sleep 1; \
done; \
alive=$$(for p in $$PIDS; do kill -0 $$p 2>/dev/null && echo $$p; done); \
if [ -n "$$alive" ]; then \
echo "Graceful shutdown did not finish in 10s, sending SIGKILL to $$alive"; \
kill -9 $$alive 2>/dev/null; \
fi; \
echo "Stopped signoz server on port $$PORT (pid $$PIDS)"
.PHONY: go-build-community $(GO_BUILD_ARCHS_COMMUNITY)
go-build-community: ## Builds the go backend server for community
go-build-community: $(GO_BUILD_ARCHS_COMMUNITY)
@@ -241,3 +266,8 @@ semconv-generate: ## Regenerate semantic-convention families for Go and TypeScri
gen-mocks:
@echo ">> Generating mocks"
@mockery --config .mockery.yml
.PHONY: gen-openapi-specs
gen-openapi-specs:
@go run cmd/enterprise/*.go generate openapi
cd frontend && pnpm generate:api && cd -

View File

@@ -1,17 +1,26 @@
# Security Policy
SigNoz is looking forward to working with security researchers across the world to keep SigNoz and our users safe. If you have found an issue in our systems/applications, please reach out to us.
SigNoz is looking forward to working with security researchers across the world to keep SigNoz and our users safe. If you have found an issue in our systems/applications, please report it to us privately.
## Supported Versions
We always recommend using the latest version of SigNoz to ensure you get all security updates
We always recommend using the latest version of SigNoz to ensure you get all security updates.
## Reporting a Vulnerability
If you believe you have found a security vulnerability within SigNoz, please let us know right away. We'll try and fix the problem as soon as possible.
**Do not report vulnerabilities using public GitHub issues**. Instead, email <security@signoz.io> with a detailed account of the issue. Please submit one issue per email, this helps us triage vulnerabilities.
**Do not report vulnerabilities using public GitHub issues, discussions, or pull requests.**
Once we've received your email we'll keep you updated as we fix the vulnerability.
Instead, report it privately through GitHub's private vulnerability reporting:
1. Go to the [**Security** tab](https://github.com/SigNoz/signoz/security) of this repository.
2. Click **Report a vulnerability**, or use [this link](https://github.com/SigNoz/signoz/security/advisories/new).
3. Describe the issue with as much detail as you can — affected version, impact, and steps to reproduce help us triage faster. Please submit one report per vulnerability.
This opens a private advisory visible only to you and the SigNoz maintainers. We'll respond there, keep you updated as we work on a fix, and coordinate disclosure. If the report is valid we'll credit you on the published advisory and request a CVE.
If you're unable to use GitHub's private reporting, you can email <security@signoz.io> instead.
## Thanks

View File

@@ -138,6 +138,12 @@ sqlstore:
##################### APIServer #####################
apiserver:
# The TCP address the API server listens on, in the form "host:port".
address: 0.0.0.0:8080
# Maximum duration for reading an entire request, including the body.
read_timeout: 60s
# Keep at 0; any value cuts off streaming endpoints (livetail, SSE, export_raw_data).
write_timeout: 0
timeout:
# Default request timeout.
default: 60s

View File

@@ -3137,6 +3137,67 @@ components:
repeatVariable:
type: string
type: object
DashboardtypesAIBuilderQuerySpec:
properties:
aggregations:
items:
$ref: '#/components/schemas/Querybuildertypesv5TraceAggregation'
nullable: true
type: array
cursor:
type: string
disabled:
type: boolean
filter:
$ref: '#/components/schemas/Querybuildertypesv5Filter'
functions:
items:
$ref: '#/components/schemas/Querybuildertypesv5Function'
nullable: true
type: array
groupBy:
items:
$ref: '#/components/schemas/Querybuildertypesv5GroupByKey'
nullable: true
type: array
having:
$ref: '#/components/schemas/Querybuildertypesv5Having'
legend:
type: string
limit:
type: integer
limitBy:
$ref: '#/components/schemas/Querybuildertypesv5LimitBy'
name:
type: string
offset:
type: integer
order:
items:
$ref: '#/components/schemas/Querybuildertypesv5OrderBy'
nullable: true
type: array
secondaryAggregations:
items:
$ref: '#/components/schemas/Querybuildertypesv5SecondaryAggregation'
nullable: true
type: array
selectFields:
items:
$ref: '#/components/schemas/TelemetrytypesTelemetryFieldKey'
nullable: true
type: array
signal:
enum:
- traces
type: string
source:
$ref: '#/components/schemas/TelemetrytypesSource'
stepInterval:
$ref: '#/components/schemas/Querybuildertypesv5Step'
required:
- signal
type: object
DashboardtypesAxes:
properties:
isLogScale:
@@ -4040,6 +4101,7 @@ components:
DashboardtypesQueryPlugin:
discriminator:
mapping:
signoz/AIBuilderQuery: '#/components/schemas/DashboardtypesQueryPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesAIBuilderQuerySpec'
signoz/BuilderQuery: '#/components/schemas/DashboardtypesQueryPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesBuilderQuerySpec'
signoz/ClickHouseSQL: '#/components/schemas/DashboardtypesQueryPluginVariantGithubComSigNozSignozPkgTypesQuerybuildertypesQuerybuildertypesv5ClickHouseQuery'
signoz/CompositeQuery: '#/components/schemas/DashboardtypesQueryPluginVariantGithubComSigNozSignozPkgTypesQuerybuildertypesQuerybuildertypesv5CompositeQuery'
@@ -4049,6 +4111,7 @@ components:
propertyName: kind
oneOf:
- $ref: '#/components/schemas/DashboardtypesQueryPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesBuilderQuerySpec'
- $ref: '#/components/schemas/DashboardtypesQueryPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesAIBuilderQuerySpec'
- $ref: '#/components/schemas/DashboardtypesQueryPluginVariantGithubComSigNozSignozPkgTypesQuerybuildertypesQuerybuildertypesv5CompositeQuery'
- $ref: '#/components/schemas/DashboardtypesQueryPluginVariantGithubComSigNozSignozPkgTypesQuerybuildertypesQuerybuildertypesv5QueryBuilderFormula'
- $ref: '#/components/schemas/DashboardtypesQueryPluginVariantGithubComSigNozSignozPkgTypesQuerybuildertypesQuerybuildertypesv5PromQuery'
@@ -4058,12 +4121,25 @@ components:
DashboardtypesQueryPluginKind:
enum:
- signoz/BuilderQuery
- signoz/AIBuilderQuery
- signoz/CompositeQuery
- signoz/Formula
- signoz/PromQLQuery
- signoz/ClickHouseSQL
- signoz/TraceOperator
type: string
DashboardtypesQueryPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesAIBuilderQuerySpec:
properties:
kind:
enum:
- signoz/AIBuilderQuery
type: string
spec:
$ref: '#/components/schemas/DashboardtypesAIBuilderQuerySpec'
required:
- kind
- spec
type: object
DashboardtypesQueryPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesBuilderQuerySpec:
properties:
kind:
@@ -9427,8 +9503,6 @@ components:
type: string
name:
type: string
origin:
$ref: '#/components/schemas/SpantypesSpanMapperOrigin'
updatedAt:
format: date-time
type: string
@@ -9441,7 +9515,6 @@ components:
- fieldContext
- config
- enabled
- origin
type: object
SpantypesSpanMapperConfig:
properties:
@@ -9470,75 +9543,48 @@ components:
type: string
orgId:
type: string
origin:
$ref: '#/components/schemas/SpantypesSpanMapperOrigin'
updatedAt:
format: date-time
type: string
updatedBy:
type: string
version:
type: integer
required:
- id
- orgId
- name
- condition
- enabled
- origin
- version
type: object
SpantypesSpanMapperGroupCondition:
nullable: true
properties:
attributes:
items:
$ref: '#/components/schemas/SpantypesSpanMapperGroupConditionKey'
type: string
nullable: true
type: array
resource:
items:
$ref: '#/components/schemas/SpantypesSpanMapperGroupConditionKey'
type: string
nullable: true
type: array
required:
- attributes
- resource
type: object
SpantypesSpanMapperGroupConditionKey:
properties:
enabled:
type: boolean
origin:
$ref: '#/components/schemas/SpantypesSpanMapperOrigin'
value:
type: string
required:
- value
- enabled
type: object
SpantypesSpanMapperOperation:
enum:
- move
- copy
type: string
SpantypesSpanMapperOrigin:
enum:
- user
- system
type: string
SpantypesSpanMapperSource:
properties:
context:
$ref: '#/components/schemas/SpantypesFieldContext'
enabled:
type: boolean
key:
type: string
operation:
$ref: '#/components/schemas/SpantypesSpanMapperOperation'
origin:
$ref: '#/components/schemas/SpantypesSpanMapperOrigin'
priority:
type: integer
required:
@@ -9546,7 +9592,6 @@ components:
- context
- operation
- priority
- enabled
type: object
SpantypesSpanMapperTestSpan:
properties:

View File

@@ -83,7 +83,13 @@ This command:
You should see: `{"status":"ok"}`
> 💡 **Tip**: The API server runs at `http://localhost:8080/` by default
3. Stop it when you're done:
```bash
make go-stop
```
> 💡 **Tip**: The API server runs at `http://localhost:8080/` by default. You can configure this using `apiserver.address` configuration option. See
> [running more than one instance](#how-do-i-run-more-than-one-instance) if you need that for agentic testing.
### 4. Setting up the Frontend
@@ -119,6 +125,36 @@ To verify everything is working correctly:
3. **Check Backend**: `curl http://localhost:8080/api/v1/health` (should return `{"status":"ok"}`)
4. **Check Frontend**: Open `http://localhost:3301` in your browser
## How do I run more than one instance?
Handy when you keep several branches checked out as separate git worktrees. Every port
and path below is read from the environment, so set them on the `make` call:
```bash
SIGNOZ_APISERVER_ADDRESS=0.0.0.0:8081 \
SIGNOZ_SQLSTORE_SQLITE_PATH=/path/to/main/sqlite.db \
SIGNOZ_INSTRUMENTATION_METRICS_READERS_PULL_EXPORTER_PROMETHEUS_PORT=9091 \
make go-run-community
```
| Variable | Default | Why you'd change it |
| --- | --- | --- |
| `SIGNOZ_APISERVER_ADDRESS` | `0.0.0.0:8080` | Address the API server listens on |
| `SIGNOZ_SQLSTORE_SQLITE_PATH` | `signoz.db` in worktree | To reuse same database |
| `SIGNOZ_INSTRUMENTATION_METRICS_READERS_PULL_EXPORTER_PROMETHEUS_PORT` | `9090` | Bound by the Prometheus metrics exporter on startup |
Point the frontend at whichever backend you want, in `frontend/.env`:
```env
VITE_FRONTEND_API_ENDPOINT=http://localhost:8081
```
Stop an instance using the address it was started on:
```bash
make go-stop SIGNOZ_APISERVER_ADDRESS=0.0.0.0:8081
```
## How to send test data?
You can now send telemetry data to your local SigNoz instance:

View File

@@ -3,56 +3,29 @@ package app
import (
"context"
"fmt"
"net"
"net/http"
"slices"
"go.opentelemetry.io/contrib/instrumentation/github.com/gorilla/mux/otelmux"
"go.opentelemetry.io/otel/propagation"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/gorilla/handlers"
"github.com/rs/cors"
"github.com/soheilhy/cmux"
"github.com/SigNoz/signoz/ee/query-service/app/api"
"github.com/SigNoz/signoz/ee/query-service/usage"
"github.com/SigNoz/signoz/pkg/http/middleware"
"github.com/SigNoz/signoz/pkg/signoz"
"github.com/SigNoz/signoz/pkg/web"
"log/slog"
"github.com/SigNoz/signoz/pkg/query-service/agentConf"
baseapp "github.com/SigNoz/signoz/pkg/query-service/app"
"github.com/SigNoz/signoz/pkg/query-service/app/clickhouseReader"
"github.com/SigNoz/signoz/pkg/query-service/app/integrations"
"github.com/SigNoz/signoz/pkg/query-service/app/logparsingpipeline"
"github.com/SigNoz/signoz/pkg/query-service/app/opamp"
opAmpModel "github.com/SigNoz/signoz/pkg/query-service/app/opamp/model"
baseconst "github.com/SigNoz/signoz/pkg/query-service/constants"
"github.com/SigNoz/signoz/pkg/query-service/healthcheck"
"github.com/SigNoz/signoz/pkg/query-service/utils"
)
// Server runs HTTP, Mux and a grpc server
// Server runs auxiliary servers (opamp) alongside the signoz apiserver
type Server struct {
config signoz.Config
signoz *signoz.SigNoz
// public http router
httpConn net.Listener
httpServer *http.Server
httpHostPort string
opampServer *opamp.Server
// Usage manager
usageManager *usage.Manager
unavailableChannel chan healthcheck.Status
}
// NewServer creates and initializes Server
@@ -127,57 +100,11 @@ func NewServer(config signoz.Config, signoz *signoz.SigNoz) (*Server, error) {
return nil, err
}
s := &Server{
config: config,
signoz: signoz,
httpHostPort: baseconst.HTTPHostPort,
unavailableChannel: make(chan healthcheck.Status),
usageManager: usageManager,
}
httpServer, err := s.createPublicServer(apiHandler, signoz.Web)
if err != nil {
return nil, err
}
s.httpServer = httpServer
s.opampServer = opamp.InitializeServer(
&opAmpModel.AllAgents, agentConfMgr, signoz.Instrumentation,
)
return s, nil
}
// HealthCheckStatus returns health check status channel a client can subscribe to
func (s Server) HealthCheckStatus() chan healthcheck.Status {
return s.unavailableChannel
}
func (s *Server) createPublicServer(apiHandler *api.APIHandler, web web.Web) (*http.Server, error) {
r := baseapp.NewRouter()
am := middleware.NewAuthZ(s.signoz.Instrumentation.Logger(), s.signoz.Modules.OrgGetter, s.signoz.Authz)
r.Use(middleware.NewRecovery(s.signoz.Instrumentation.Logger()).Wrap)
r.Use(otelmux.Middleware(
"apiserver",
otelmux.WithMeterProvider(s.signoz.Instrumentation.MeterProvider()),
otelmux.WithTracerProvider(s.signoz.Instrumentation.TracerProvider()),
otelmux.WithPropagators(propagation.NewCompositeTextMapPropagator(propagation.Baggage{}, propagation.TraceContext{})),
otelmux.WithFilter(func(r *http.Request) bool {
return !slices.Contains([]string{"/api/v1/health"}, r.URL.Path)
}),
))
r.Use(middleware.NewIdentN(s.signoz.IdentNResolver, s.signoz.Sharder, s.signoz.Instrumentation.Logger()).Wrap)
r.Use(middleware.NewTimeout(s.signoz.Instrumentation.Logger(),
s.config.APIServer.Timeout.ExcludedRoutes,
s.config.APIServer.Timeout.Default,
s.config.APIServer.Timeout.Max,
).Wrap)
r.Use(middleware.NewResource(s.signoz.Instrumentation.Logger()).Wrap)
r.Use(middleware.NewAudit(s.signoz.Instrumentation.Logger(), s.config.APIServer.Logging.ExcludedRoutes, s.signoz.Auditor).Wrap)
r.Use(middleware.NewComment().Wrap)
// Register the legacy query-service routes on the apiserver router. The
// apiserver owns the HTTP server and applies the middleware chain at serve
// time, so these routes get the same treatment as the apiserver routes.
r := signoz.APIServer.Router()
am := middleware.NewAuthZ(signoz.Instrumentation.Logger(), signoz.Modules.OrgGetter, signoz.Authz)
apiHandler.RegisterRoutes(r, am)
apiHandler.RegisterLogsRoutes(r, am)
@@ -188,107 +115,29 @@ func (s *Server) createPublicServer(apiHandler *api.APIHandler, web web.Web) (*h
apiHandler.RegisterThirdPartyApiRoutes(r, am)
apiHandler.RegisterTraceFunnelsRoutes(r, am)
err := s.signoz.APIServer.AddToRouter(r)
if err != nil {
return nil, err
s := &Server{
usageManager: usageManager,
}
c := cors.New(cors.Options{
AllowedOrigins: []string{"*"},
AllowedMethods: []string{"GET", "DELETE", "POST", "PUT", "PATCH", "OPTIONS"},
AllowedHeaders: []string{"Accept", "Authorization", "Content-Type", "cache-control"},
})
s.opampServer = opamp.InitializeServer(
&opAmpModel.AllAgents, agentConfMgr, signoz.Instrumentation,
)
handler := c.Handler(r)
handler = handlers.CompressHandler(handler)
err = web.AddToRouter(r)
if err != nil {
return nil, err
}
routePrefix := s.config.Global.ExternalPath()
if routePrefix != "" {
prefixed := http.StripPrefix(routePrefix, handler)
handler = http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
switch req.URL.Path {
case "/api/v1/health", "/api/v2/healthz", "/api/v2/readyz", "/api/v2/livez":
r.ServeHTTP(w, req)
return
}
prefixed.ServeHTTP(w, req)
})
}
return &http.Server{
Handler: handler,
}, nil
return s, nil
}
// initListeners initialises listeners of the server
func (s *Server) initListeners() error {
// listen on public port
var err error
publicHostPort := s.httpHostPort
if publicHostPort == "" {
return fmt.Errorf("baseconst.HTTPHostPort is required")
}
s.httpConn, err = net.Listen("tcp", publicHostPort)
if err != nil {
return err
}
slog.Info(fmt.Sprintf("Query server started listening on %s...", s.httpHostPort))
return nil
}
// Start listening on http and private http port concurrently
// Start starts the opamp websocket server. The HTTP API server is started by
// the signoz registry.
func (s *Server) Start(ctx context.Context) error {
err := s.initListeners()
if err != nil {
slog.Info("Starting OpAmp Websocket server", "addr", baseconst.OpAmpWsEndpoint)
if err := s.opampServer.Start(baseconst.OpAmpWsEndpoint); err != nil {
return err
}
var httpPort int
if port, err := utils.GetPort(s.httpConn.Addr()); err == nil {
httpPort = port
}
go func() {
slog.Info("Starting HTTP server", "port", httpPort, "addr", s.httpHostPort)
switch err := s.httpServer.Serve(s.httpConn); err {
case nil, http.ErrServerClosed, cmux.ErrListenerClosed:
// normal exit, nothing to do
default:
slog.Error("Could not start HTTP server", errors.Attr(err))
}
s.unavailableChannel <- healthcheck.Unavailable
}()
go func() {
slog.Info("Starting OpAmp Websocket server", "addr", baseconst.OpAmpWsEndpoint)
err := s.opampServer.Start(baseconst.OpAmpWsEndpoint)
if err != nil {
slog.Error("opamp ws server failed to start", errors.Attr(err))
s.unavailableChannel <- healthcheck.Unavailable
}
}()
return nil
}
func (s *Server) Stop(ctx context.Context) error {
if s.httpServer != nil {
if err := s.httpServer.Shutdown(ctx); err != nil {
return err
}
}
s.opampServer.Stop()
// stop usage manager

View File

@@ -10518,31 +10518,15 @@ export interface SpantypesGettableFlamegraphTraceDTO {
startTimestampMillis: number;
}
export enum SpantypesSpanMapperOriginDTO {
user = 'user',
system = 'system',
}
export interface SpantypesSpanMapperGroupConditionKeyDTO {
/**
* @type boolean
*/
enabled: boolean;
origin?: SpantypesSpanMapperOriginDTO;
/**
* @type string
*/
value: string;
}
export type SpantypesSpanMapperGroupConditionDTOAnyOf = {
/**
* @type array,null
*/
attributes: SpantypesSpanMapperGroupConditionKeyDTO[] | null;
attributes: string[] | null;
/**
* @type array,null
*/
resource: SpantypesSpanMapperGroupConditionKeyDTO[] | null;
resource: string[] | null;
};
/**
@@ -10578,7 +10562,6 @@ export interface SpantypesSpanMapperGroupDTO {
* @type string
*/
orgId: string;
origin: SpantypesSpanMapperOriginDTO;
/**
* @type string
* @format date-time
@@ -10588,10 +10571,6 @@ export interface SpantypesSpanMapperGroupDTO {
* @type string
*/
updatedBy?: string;
/**
* @type integer
*/
version: number;
}
export interface SpantypesGettableSpanMapperGroupsDTO {
@@ -10649,16 +10628,11 @@ export enum SpantypesSpanMapperOperationDTO {
}
export interface SpantypesSpanMapperSourceDTO {
context: SpantypesFieldContextDTO;
/**
* @type boolean
*/
enabled: boolean;
/**
* @type string
*/
key: string;
operation: SpantypesSpanMapperOperationDTO;
origin?: SpantypesSpanMapperOriginDTO;
/**
* @type integer
*/
@@ -10700,7 +10674,6 @@ export interface SpantypesSpanMapperDTO {
* @type string
*/
name: string;
origin: SpantypesSpanMapperOriginDTO;
/**
* @type string
* @format date-time

View File

@@ -333,7 +333,6 @@ describe('AttributeMappingsTab (integration)', () => {
context: FieldContext.attribute,
operation: MapperOperation.copy,
priority,
enabled: true,
})),
},
}),

View File

@@ -1,12 +1,10 @@
import { Typography } from '@signozhq/ui/typography';
import { ConditionKey } from 'container/LLMObservability/AttributeMapping/types';
import styles from './ConditionsTooltip.module.scss';
interface ConditionsTooltipProps {
attributes: ConditionKey[];
resource: ConditionKey[];
attributes: string[];
resource: string[];
}
function ConditionsTooltip({
@@ -35,8 +33,8 @@ function ConditionsTooltip({
</Typography.Text>
<div className={styles.keyList}>
{attributes.map((key) => (
<code key={`${key.origin}-${key.value}`} className={styles.key}>
{key.value}
<code key={key} className={styles.key}>
{key}
</code>
))}
</div>
@@ -49,8 +47,8 @@ function ConditionsTooltip({
</Typography.Text>
<div className={styles.keyList}>
{resource.map((key) => (
<code key={`${key.origin}-${key.value}`} className={styles.key}>
{key.value}
<code key={key} className={styles.key}>
{key}
</code>
))}
</div>

View File

@@ -3,7 +3,6 @@ import {
SpantypesSpanMapperDTO as Mapper,
SpantypesSpanMapperGroupDTO as MapperGroup,
SpantypesSpanMapperOperationDTO as MapperOperation,
SpantypesSpanMapperOriginDTO as MapperOrigin,
SpantypesSpanMapperTestSpanDTO as TestSpan,
} from 'api/generated/services/sigNoz.schemas';
@@ -22,15 +21,9 @@ export function makeGroup(overrides: Partial<MapperGroup> = {}): MapperGroup {
orgId: 'org-1',
name: 'demo',
enabled: true,
origin: MapperOrigin.user,
version: 0,
condition: {
attributes: [
{ value: 'ai.embeddings', enabled: true, origin: MapperOrigin.user },
],
resource: [
{ value: 'cloud.account.id', enabled: true, origin: MapperOrigin.user },
],
attributes: ['ai.embeddings'],
resource: ['cloud.account.id'],
},
...overrides,
};
@@ -42,7 +35,6 @@ export function makeMapper(overrides: Partial<Mapper> = {}): Mapper {
groupId: 'group-1',
name: 'gen_ai.request.model',
enabled: true,
origin: MapperOrigin.user,
fieldContext: FieldContext.attribute,
config: {
sources: [
@@ -51,16 +43,12 @@ export function makeMapper(overrides: Partial<Mapper> = {}): Mapper {
context: FieldContext.attribute,
operation: MapperOperation.copy,
priority: 2,
enabled: true,
origin: MapperOrigin.user,
},
{
key: 'llm.model',
context: FieldContext.attribute,
operation: MapperOperation.move,
priority: 1,
enabled: true,
origin: MapperOrigin.user,
},
],
},
@@ -97,12 +85,8 @@ export const mockGroups: MapperGroup[] = [
id: 'group-1',
name: 'demo',
condition: {
attributes: [
{ value: 'ai.embeddings', enabled: true, origin: MapperOrigin.user },
],
resource: [
{ value: 'cloud.account.id', enabled: true, origin: MapperOrigin.user },
],
attributes: ['ai.embeddings'],
resource: ['cloud.account.id'],
},
}),
makeGroup({

View File

@@ -1,23 +1,19 @@
import { Button } from '@signozhq/ui/button';
import { Plus, X } from '@signozhq/icons';
import {
ConditionKey,
FieldContextValue,
} from 'container/LLMObservability/AttributeMapping/types';
import { createConditionKey } from 'container/LLMObservability/AttributeMapping/utils';
import { FieldContextValue } from 'container/LLMObservability/AttributeMapping/types';
import KeySearchInput from '../../../KeySearchInput/KeySearchInput';
import styles from './ConditionKeyList.module.scss';
interface ConditionKeyListProps {
label: string;
labelHint?: string;
keys: ConditionKey[];
keys: string[];
placeholder: string;
addLabel: string;
testIdPrefix: string;
fieldContext: FieldContextValue;
onChange: (keys: ConditionKey[]) => void;
onChange: (keys: string[]) => void;
}
function ConditionKeyList({
@@ -31,11 +27,11 @@ function ConditionKeyList({
onChange,
}: ConditionKeyListProps): JSX.Element {
const updateKey = (index: number, value: string): void => {
onChange(keys.map((key, i) => (i === index ? { ...key, value } : key)));
onChange(keys.map((key, i) => (i === index ? value : key)));
};
const addKey = (): void => {
onChange([...keys, createConditionKey()]);
onChange([...keys, '']);
};
const removeKey = (index: number): void => {
@@ -57,7 +53,7 @@ function ConditionKeyList({
<KeySearchInput
className={styles.keyInput}
placeholder={placeholder}
value={key.value}
value={key}
fieldContext={fieldContext}
onChange={(next): void => updateKey(index, next)}
testId={`${testIdPrefix}-${index}`}

View File

@@ -42,9 +42,7 @@ function sourcesEqual(a: SourceConfig[], b: SourceConfig[]): boolean {
(source, index) =>
source.key === b[index].key &&
source.context === b[index].context &&
source.operation === b[index].operation &&
source.enabled === b[index].enabled &&
source.origin === b[index].origin,
source.operation === b[index].operation,
)
);
}

View File

@@ -3,7 +3,6 @@ import {
SpantypesSpanMapperDTO,
SpantypesSpanMapperGroupDTO,
SpantypesSpanMapperOperationDTO,
SpantypesSpanMapperOriginDTO,
} from 'api/generated/services/sigNoz.schemas';
export type MapperGroup = SpantypesSpanMapperGroupDTO;
@@ -12,16 +11,6 @@ export const FieldContext = SpantypesFieldContextDTO;
export type FieldContextValue = SpantypesFieldContextDTO;
export const MapperOperation = SpantypesSpanMapperOperationDTO;
export type MapperOperationValue = SpantypesSpanMapperOperationDTO;
export const MapperOrigin = SpantypesSpanMapperOriginDTO;
export type MapperOriginValue = SpantypesSpanMapperOriginDTO;
// One condition substring. Shipped (system) keys are read-only apart from
// `enabled`; user keys are fully editable.
export interface ConditionKey {
value: string;
enabled: boolean;
origin: MapperOriginValue;
}
export type MapperDraftMode = 'add' | 'edit';
@@ -29,8 +18,6 @@ export interface SourceConfig {
key: string;
context: SpantypesFieldContextDTO;
operation: SpantypesSpanMapperOperationDTO;
enabled: boolean;
origin: MapperOriginValue;
}
// Editable form state for a mapper. `sources` is ordered highest priority
@@ -46,8 +33,8 @@ export interface MapperDraft {
export interface GroupDraft {
id: string | null;
name: string;
attributes: ConditionKey[];
resource: ConditionKey[];
attributes: string[];
resource: string[];
enabled: boolean;
}
@@ -64,8 +51,8 @@ export interface DraftGroup {
localId: string;
serverId: string | null;
name: string;
attributes: ConditionKey[];
resource: ConditionKey[];
attributes: string[];
resource: string[];
enabled: boolean;
mappers: DraftMapper[];
}

View File

@@ -1,14 +1,12 @@
import {
SpantypesPostableSpanMapperDTO,
SpantypesPostableSpanMapperGroupDTO,
SpantypesSpanMapperGroupConditionKeyDTO,
SpantypesUpdatableSpanMapperDTO,
SpantypesUpdatableSpanMapperGroupDTO,
} from 'api/generated/services/sigNoz.schemas';
import { v4 as uuid } from 'uuid';
import {
ConditionKey,
DraftGroup,
DraftMapper,
FieldContext,
@@ -17,7 +15,6 @@ import {
MapperDraft,
MapperGroup,
MapperOperation,
MapperOrigin,
SourceConfig,
} from './types';
@@ -27,36 +24,20 @@ function genLocalId(prefix: 'group' | 'mapper'): string {
return `local-${prefix}-${uuid()}`;
}
export function createConditionKey(value = ''): ConditionKey {
return { value, enabled: true, origin: MapperOrigin.user };
}
// Trimmed, de-duplicated, non-empty keys preserving input order. A shipped and
// a user key may share a value, so the origin is part of the identity.
function cleanKeys(keys: ConditionKey[]): ConditionKey[] {
// Trimmed, de-duplicated, non-empty keys preserving input order.
function cleanKeys(keys: string[]): string[] {
const seen = new Set<string>();
const result: ConditionKey[] = [];
const result: string[] = [];
keys.forEach((raw) => {
const value = raw.value.trim();
const dedupeKey = `${raw.origin}:${value}`;
if (value && !seen.has(dedupeKey)) {
seen.add(dedupeKey);
result.push({ ...raw, value });
const key = raw.trim();
if (key && !seen.has(key)) {
seen.add(key);
result.push(key);
}
});
return result;
}
function fromConditionKeys(
keys: SpantypesSpanMapperGroupConditionKeyDTO[] | null | undefined,
): ConditionKey[] {
return (keys ?? []).map((key) => ({
value: key.value,
enabled: key.enabled,
origin: key.origin ?? MapperOrigin.user,
}));
}
// Source configs for a mapper, highest priority first (first match wins at
// evaluation time).
function getMapperSources(mapper: Mapper): SourceConfig[] {
@@ -67,8 +48,6 @@ function getMapperSources(mapper: Mapper): SourceConfig[] {
key: source.key,
context: source.context,
operation: source.operation,
enabled: source.enabled,
origin: source.origin ?? MapperOrigin.user,
}));
}
@@ -77,8 +56,6 @@ export function createEmptySource(): SourceConfig {
key: '',
context: FieldContext.attribute,
operation: MapperOperation.copy,
enabled: true,
origin: MapperOrigin.user,
};
}
@@ -95,7 +72,7 @@ function getCleanSources(draft: MapperDraft): SourceConfig[] {
const result: SourceConfig[] = [];
draft.sources.forEach((source) => {
const key = source.key.trim();
const dedupeKey = `${source.origin}:${source.context}:${key}`;
const dedupeKey = `${source.context}:${key}`;
if (key && !seen.has(dedupeKey)) {
seen.add(dedupeKey);
result.push({ ...source, key });
@@ -118,8 +95,6 @@ function buildSources(
context: source.context,
operation: source.operation,
priority: sources.length - index,
enabled: source.enabled,
origin: source.origin,
}));
}
@@ -148,7 +123,7 @@ export function buildUpdatableMapper(
export const EMPTY_GROUP_DRAFT: GroupDraft = {
id: null,
name: '',
attributes: [createConditionKey()],
attributes: [''],
resource: [],
enabled: true,
};
@@ -195,8 +170,8 @@ export function buildDraftGroup(
localId: group.id,
serverId: group.id,
name: group.name,
attributes: fromConditionKeys(group.condition?.attributes),
resource: fromConditionKeys(group.condition?.resource),
attributes: group.condition?.attributes ?? [],
resource: group.condition?.resource ?? [],
enabled: group.enabled,
mappers: mappers.map(buildDraftMapper),
};
@@ -207,8 +182,7 @@ export function groupDraftFromNode(group: DraftGroup): GroupDraft {
return {
id: group.localId,
name: group.name,
attributes:
group.attributes.length > 0 ? group.attributes : [createConditionKey()],
attributes: group.attributes.length > 0 ? group.attributes : [''],
resource: group.resource,
enabled: group.enabled,
};

View File

@@ -0,0 +1,14 @@
.container {
display: flex;
align-items: center;
justify-content: flex-end;
gap: 0.3rem;
margin: 8px 0;
}
.optionsTrigger {
display: flex;
align-items: center;
gap: 4px;
cursor: pointer;
}

View File

@@ -0,0 +1,82 @@
import { memo, useState } from 'react';
import { useTranslation } from 'react-i18next';
import { Settings } from '@signozhq/icons';
import FieldsSelector from 'components/FieldsSelector';
import Controls, { ControlsProps } from 'container/Controls';
import { OptionsMenuConfig } from 'container/OptionsMenu/types';
import useQueryPagination from 'hooks/queryPagination/useQueryPagination';
import { DataSource } from 'types/common/queryBuilder';
import styles from './Controls.module.scss';
function TraceExplorerControls({
isLoading,
totalCount,
perPageOptions,
config,
showSizeChanger = true,
}: TraceExplorerControlsProps): JSX.Element | null {
const { t } = useTranslation(['trace']);
const [isFieldsSelectorOpen, setIsFieldsSelectorOpen] = useState(false);
const {
pagination,
handleCountItemsPerPageChange,
handleNavigateNext,
handleNavigatePrevious,
} = useQueryPagination(totalCount, perPageOptions);
return (
<div className={styles.container}>
{config?.fieldsSelector && (
<>
<div
className={styles.optionsTrigger}
onClick={(): void => setIsFieldsSelectorOpen(true)}
>
{t('options_menu.options')}
<Settings size="md" />
</div>
<FieldsSelector
isOpen={isFieldsSelectorOpen}
title="Edit columns"
fields={config.fieldsSelector.value}
onFieldsChange={config.fieldsSelector.onFieldsChange}
onClose={(): void => setIsFieldsSelectorOpen(false)}
signal={DataSource.TRACES}
/>
</>
)}
<Controls
isLoading={isLoading}
totalCount={totalCount}
offset={pagination.offset}
countPerPage={pagination.limit}
perPageOptions={perPageOptions}
handleCountItemsPerPageChange={handleCountItemsPerPageChange}
handleNavigateNext={handleNavigateNext}
handleNavigatePrevious={handleNavigatePrevious}
showSizeChanger={showSizeChanger}
/>
</div>
);
}
TraceExplorerControls.defaultProps = {
config: null,
};
type TraceExplorerControlsProps = Pick<
ControlsProps,
'isLoading' | 'totalCount' | 'perPageOptions'
> & {
config?: OptionsMenuConfig | null;
showSizeChanger?: boolean;
};
TraceExplorerControls.defaultProps = {
showSizeChanger: true,
};
export default memo(TraceExplorerControls);

View File

@@ -0,0 +1,168 @@
import { Link } from 'react-router-dom';
import type { TableColumnsType as ColumnsType } from 'antd';
import { Badge } from '@signozhq/ui/badge';
import { Typography } from '@signozhq/ui/typography';
import { TelemetryFieldKey } from 'api/v5/v5';
import type { TracesTableRow } from '../TracesTable/getFieldColumn';
import { DATE_TIME_FORMATS } from 'constants/dateTimeFormats';
import ROUTES from 'constants/routes';
import { buildCompositeKey } from 'container/OptionsMenu/utils';
import { getMs } from 'container/Trace/Filters/Panel/PanelBody/Duration/util';
import { formUrlParams } from 'container/TraceDetail/utils';
import { TimestampInput } from 'hooks/useTimezoneFormatter/useTimezoneFormatter';
import { RowData } from 'lib/query/createTableColumnsFromQuery';
import LineClampedText from 'periscope/components/LineClampedText/LineClampedText';
import { ILog } from 'types/api/logs/log';
import { QueryDataV3 } from 'types/api/widgets/getQuery';
export function BlockLink({
children,
to,
openInNewTab,
}: {
children: React.ReactNode;
to: string;
openInNewTab: boolean;
}): any {
// Display block to make the whole cell clickable
return (
<Link
to={to}
style={{ display: 'block' }}
target={openInNewTab ? '_blank' : '_self'}
>
{children}
</Link>
);
}
export const transformDataWithDate = (
data: QueryDataV3[],
): Omit<ILog, 'timestamp'>[] =>
data[0]?.list?.map(({ data, timestamp }) => ({ ...data, date: timestamp })) ||
[];
export const getTraceLink = (record: Record<string, unknown>): string => {
function readId(value: unknown): string {
if (typeof value === 'string' || typeof value === 'number') {
return String(value);
}
return '';
}
const traceId = readId(record.traceID) || readId(record.trace_id);
const spanId = readId(record.spanID) || readId(record.span_id);
return `${ROUTES.TRACE}/${traceId}${formUrlParams({
spanId,
levelUp: 0,
levelDown: 0,
})}`;
};
export const getListColumns = (
selectedColumns: TelemetryFieldKey[],
formatTimezoneAdjustedTimestamp: (
input: TimestampInput,
format?: string,
) => string | number,
): ColumnsType<RowData> => {
const initialColumns: ColumnsType<RowData> = [
{
dataIndex: 'date',
key: 'date',
title: 'Timestamp',
width: 145,
render: (value, item): JSX.Element => {
const date =
typeof value === 'string'
? formatTimezoneAdjustedTimestamp(
value,
DATE_TIME_FORMATS.ISO_DATETIME_MS,
)
: formatTimezoneAdjustedTimestamp(
value / 1e6,
DATE_TIME_FORMATS.ISO_DATETIME_MS,
);
return (
<BlockLink to={getTraceLink(item)} openInNewTab={false}>
<Typography.Text>{date}</Typography.Text>
</BlockLink>
);
},
},
];
const columns: ColumnsType<RowData> =
selectedColumns.map((props) => {
const name = props?.name || (props as any)?.key;
const fieldContext = props?.fieldContext || (props as any)?.type;
return {
title: name,
dataIndex: name,
key: buildCompositeKey(name, fieldContext),
width: 145,
render: (value, item): JSX.Element => {
if (value === '') {
return (
<BlockLink to={getTraceLink(item)} openInNewTab={false}>
<Typography data-testid={name}>N/A</Typography>
</BlockLink>
);
}
if (
name === 'httpMethod' ||
name === 'responseStatusCode' ||
name === 'response_status_code' ||
name === 'http_method'
) {
return (
<BlockLink to={getTraceLink(item)} openInNewTab={false}>
<Badge data-testid={name} color="sakura" variant="outline">
{value}
</Badge>
</BlockLink>
);
}
if (name === 'durationNano' || name === 'duration_nano') {
return (
<BlockLink to={getTraceLink(item)} openInNewTab={false}>
<Typography data-testid={name}>{getMs(value)}ms</Typography>
</BlockLink>
);
}
return (
<BlockLink to={getTraceLink(item)} openInNewTab={false}>
<Typography data-testid={name}>
<LineClampedText text={value} lines={3} />
</Typography>
</BlockLink>
);
},
responsive: ['md'],
};
}) || [];
return [...initialColumns, ...columns];
};
// Reshapes the query-range list payload into table rows. `id` mirrors span_id so
// TanStack sees genuine row changes on orderBy toggles instead of falling back to
// positional ids; `timestamp` is lifted from the wrapping ListItem.
export const transformSpanRows = (data: QueryDataV3[]): TracesTableRow[] => {
const list = data[0]?.list;
if (!list) {
return [];
}
return list.map((item) => {
const row = item.data as Record<string, unknown>;
return {
...row,
timestamp: item.timestamp,
id: row.span_id,
};
}) as TracesTableRow[];
};

View File

@@ -0,0 +1,19 @@
.loading-traces {
padding: 24px 0;
height: 240px;
display: flex;
justify-content: center;
align-items: flex-start;
.loading-traces-content {
display: flex;
align-items: flex-start;
flex-direction: column;
.loading-gif {
height: 72px;
margin-left: -24px;
}
}
}

View File

@@ -0,0 +1,22 @@
import { useTranslation } from 'react-i18next';
import { Typography } from '@signozhq/ui/typography';
import { DataSource } from 'types/common/queryBuilder';
import loadingPlaneUrl from '@/assets/Icons/loading-plane.gif';
import './TraceLoading.styles.scss';
export function TracesLoading(): JSX.Element {
const { t } = useTranslation('common');
return (
<div className="loading-traces">
<div className="loading-traces-content">
<img className="loading-gif" src={loadingPlaneUrl} alt="wait-icon" />
<Typography>
{t('pending_data_placeholder', { dataSource: DataSource.TRACES })}
</Typography>
</div>
</div>
);
}

View File

@@ -0,0 +1,77 @@
import { generatePath, Link } from 'react-router-dom';
import { Badge } from '@signozhq/ui/badge';
import TanStackTable from 'components/TanStackTableView';
import { DATE_TIME_FORMATS } from 'constants/dateTimeFormats';
import ROUTES from 'constants/routes';
import { getMs } from 'container/Trace/Filters/Panel/PanelBody/Duration/util';
import { useTimezone } from 'providers/Timezone';
import {
DURATION_FIELD_NAMES,
STATUS_FIELD_NAMES,
TIMESTAMP_FIELD_NAMES,
TRACE_ID_FIELD_NAMES,
} from './constants';
import { stringifyCellValue } from './utils';
type FieldCellProps = {
name: string;
value: unknown;
};
function FieldCell({ name, value }: FieldCellProps): JSX.Element {
const { formatTimezoneAdjustedTimestamp } = useTimezone();
if (TIMESTAMP_FIELD_NAMES.has(name)) {
const ts = value as string | number;
const formatted =
typeof ts === 'string'
? formatTimezoneAdjustedTimestamp(ts, DATE_TIME_FORMATS.ISO_DATETIME_MS)
: formatTimezoneAdjustedTimestamp(
ts / 1e6,
DATE_TIME_FORMATS.ISO_DATETIME_MS,
);
const text = String(formatted);
return <TanStackTable.Text title={text}>{text}</TanStackTable.Text>;
}
if (value === '' || value == null) {
return <TanStackTable.Text data-testid={name}>-</TanStackTable.Text>;
}
const text = stringifyCellValue(value);
if (TRACE_ID_FIELD_NAMES.has(name)) {
return (
<Link
to={generatePath(ROUTES.TRACE_DETAIL, { id: text })}
data-testid="trace-id"
onClick={(e): void => e.stopPropagation()}
>
{text}
</Link>
);
}
if (STATUS_FIELD_NAMES.has(name)) {
return (
<Badge data-testid={name} color="sakura" variant="outline">
{text}
</Badge>
);
}
if (DURATION_FIELD_NAMES.has(name)) {
return (
<TanStackTable.Text data-testid={name}>{getMs(text)}ms</TanStackTable.Text>
);
}
return (
<TanStackTable.Text data-testid={name} title={text}>
{text}
</TanStackTable.Text>
);
}
export default FieldCell;

View File

@@ -0,0 +1,26 @@
.tableWrapper {
flex: 1;
min-height: 0;
display: flex;
flex-direction: column;
}
.tracesTable {
--tanstack-table-row-height: 54px;
--tanstack-table-header-height: 54px;
--tanstack-cell-padding-top-override: 5px;
--tanstack-cell-padding-bottom-override: 5px;
--tanstack-cell-padding-right-override: 15px;
--tanstack-cell-padding-left-override: 15px;
--tanstack-cell-header-padding-left-override: 5px;
--tanstack-cell-header-padding-left-first-column: 15px;
--tanstack-plain-body-line-clamp: 1;
--tanstack-table-cell-bg: var(--l2-background);
--tanstack-table-header-cell-bg: var(--l1-background-hover);
--tanstack-table-row-hover-bg: var(--l1-background-hover);
}

View File

@@ -0,0 +1,116 @@
import { useCallback } from 'react';
import { useHistory } from 'react-router-dom';
import ErrorInPlace from 'components/ErrorInPlace/ErrorInPlace';
import TanStackTable from 'components/TanStackTableView';
import type {
CellTypographySize,
TableColumnDef,
} from 'components/TanStackTableView/types';
import EmptyLogsSearch from 'container/EmptyLogsSearch/EmptyLogsSearch';
import NoLogs from 'container/NoLogs/NoLogs';
import { TracesLoading } from '../TraceLoading/TraceLoading';
import APIError from 'types/api/error';
import { DataSource, PanelTypeKeys } from 'types/common/queryBuilder';
import { getAbsoluteUrl } from 'utils/basePath';
import type { TracesTableRow } from './getFieldColumn';
import styles from './TracesTable.module.scss';
export type TracesTableProps = {
data: TracesTableRow[];
columns: TableColumnDef<TracesTableRow>[];
columnStorageKey?: string;
respectColumnOrder?: boolean;
panelType: PanelTypeKeys;
/** Builds the trace-detail href for a row; drives row click + cmd/ctrl-click. */
getRowHref: (row: TracesTableRow) => string;
isLoading: boolean;
isFetching: boolean;
isError: boolean;
error: APIError | Error | null;
isFilterApplied: boolean;
onColumnOrderChange?: (cols: TableColumnDef<TracesTableRow>[]) => void;
onColumnRemove?: (columnId: string) => void;
cellTypographySize?: CellTypographySize;
};
function TracesTable({
data,
columns,
columnStorageKey,
respectColumnOrder = false,
panelType,
getRowHref,
isLoading,
isFetching,
isError,
error,
isFilterApplied,
onColumnOrderChange,
onColumnRemove,
cellTypographySize = 'medium',
}: TracesTableProps): JSX.Element {
const history = useHistory();
const isDataAbsent =
!isLoading && !isFetching && !isError && data.length === 0;
const handleRowClick = useCallback(
(row: TracesTableRow): void => {
history.push(getRowHref(row));
},
[history, getRowHref],
);
const handleRowClickNewTab = useCallback(
(row: TracesTableRow): void => {
window.open(getAbsoluteUrl(getRowHref(row)), '_blank', 'noopener');
},
[getRowHref],
);
return (
<>
{isError && error && <ErrorInPlace error={error as APIError} />}
{(isLoading || (isFetching && data.length === 0)) && <TracesLoading />}
{isDataAbsent && !isFilterApplied && (
<NoLogs dataSource={DataSource.TRACES} />
)}
{isDataAbsent && isFilterApplied && (
<EmptyLogsSearch dataSource={DataSource.TRACES} panelType={panelType} />
)}
{!isError && data.length !== 0 && (
<div className={styles.tableWrapper}>
<TanStackTable<TracesTableRow>
data={data}
columns={columns}
className={styles.tracesTable}
columnStorageKey={columnStorageKey}
respectColumnOrder={respectColumnOrder}
isLoading={isFetching}
cellTypographySize={cellTypographySize}
onColumnOrderChange={onColumnOrderChange}
onColumnRemove={onColumnRemove}
onRowClick={handleRowClick}
onRowClickNewTab={handleRowClickNewTab}
getRowTestId={(row): string => `traces-table-row-${row.id}`}
/>
</div>
)}
</>
);
}
TracesTable.defaultProps = {
columnStorageKey: undefined,
respectColumnOrder: false,
onColumnOrderChange: undefined,
onColumnRemove: undefined,
cellTypographySize: 'medium',
};
export default TracesTable;

View File

@@ -0,0 +1,18 @@
// Field-name allowlists that drive signal-specific cell rendering. Both legacy
// camelCase and snake_case variants are listed because the API has shipped both.
export const TIMESTAMP_FIELD_NAMES = new Set(['timestamp']);
export const STATUS_FIELD_NAMES = new Set([
'httpMethod',
'http_method',
'http.method',
'http.request.method',
'responseStatusCode',
'response_status_code',
'http.status_code',
'http.response.status_code',
]);
export const DURATION_FIELD_NAMES = new Set(['durationNano', 'duration_nano']);
export const TRACE_ID_FIELD_NAMES = new Set(['traceID', 'trace_id']);

View File

@@ -0,0 +1,26 @@
import { TelemetryFieldKey } from 'api/v5/v5';
import type { TableColumnDef } from 'components/TanStackTableView/types';
import { buildCompositeKey } from 'container/OptionsMenu/utils';
import { TIMESTAMP_FIELD_NAMES } from './constants';
import FieldCell from './FieldCell';
export type TracesTableRow = { id: string } & Record<string, unknown>;
export function getFieldColumn(
field: TelemetryFieldKey,
): TableColumnDef<TracesTableRow> {
const { name, fieldContext, fieldDataType } = field;
const isTimestamp = TIMESTAMP_FIELD_NAMES.has(name);
return {
id: buildCompositeKey(name, fieldContext, fieldDataType),
header: name,
accessorFn: (row): unknown => row[name],
enableMove: !isTimestamp,
enableRemove: !isTimestamp,
canBeHidden: !isTimestamp,
width: { min: 192 },
cell: ({ value }): JSX.Element => <FieldCell name={name} value={value} />,
};
}

View File

@@ -0,0 +1,12 @@
export function stringifyCellValue(value: unknown): string {
if (value == null) {
return '';
}
if (typeof value === 'string') {
return value;
}
if (typeof value === 'number' || typeof value === 'boolean') {
return String(value);
}
return JSON.stringify(value);
}

View File

@@ -1,9 +1,6 @@
import { TelemetryFieldKey } from 'api/v5/v5';
import type { TableColumnDef } from 'components/TanStackTableView/types';
import {
getFieldColumn,
TracesTableRow,
} from 'container/TracesExplorer/TracesTable/getFieldColumn';
import { getFieldColumn, TracesTableRow } from '../TracesTable/getFieldColumn';
import { DEFAULT_PER_PAGE_OPTIONS } from 'hooks/queryPagination';
export const PER_PAGE_OPTIONS: number[] = [10, ...DEFAULT_PER_PAGE_OPTIONS];

View File

@@ -0,0 +1,235 @@
/**
* AI Assistant page-action factories for the Traces Explorer.
*
* Mirrors the logs equivalents — each factory closes over live page
* state/callbacks so `execute()` always operates on the current query, and
* the page component instantiates them via `useMemo` + `usePageActions`.
*
* See `pages/LogsExplorer/aiActions.ts` for the rationale behind writing
* BOTH `filters.items` and `filter.expression` and then re-using the same
* URL parser shape via `redirectWithQueryBuilderData`.
*/
import { convertFiltersToExpression } from 'components/QueryBuilderV2/utils';
import {
aiFilterToTagFilterItem,
FILTER_OP_ENUM,
FILTER_VALUE_DESCRIPTION,
FilterDeps,
replaceFirstQueryData,
} from 'container/AIAssistant/pageActions/builderQueryHelpers';
import {
ActionResult,
PageAction,
} from 'container/AIAssistant/pageActions/types';
import {
IBuilderQuery,
TagFilterItem,
} from 'types/api/queryBuilder/queryBuilderData';
interface AIFilter {
key: string;
op: string;
value: string;
}
interface RunQueryParams {
filters: AIFilter[];
}
interface AddFilterParams {
key: string;
op: string;
value: string;
}
type TracesView = 'list' | 'timeseries' | 'table' | 'trace';
interface ChangeViewParams {
view: TracesView;
}
interface SaveViewParams {
name: string;
}
/**
* Replace all active span filters and navigate to the updated query URL
* (which makes the WHERE clause reflect the new filters and triggers a re-run).
*/
export function tracesRunQueryAction(
deps: FilterDeps,
): PageAction<RunQueryParams> {
return {
id: 'traces.runQuery',
description: 'Replace the active trace filters and re-run the query',
parameters: {
type: 'object',
properties: {
filters: {
type: 'array',
description: 'Replacement filter list',
items: {
type: 'object',
properties: {
key: {
type: 'string',
description: 'Attribute key, e.g. service.name, http.status_code',
},
op: {
type: 'string',
enum: [...FILTER_OP_ENUM],
},
value: {
type: 'string',
description: FILTER_VALUE_DESCRIPTION,
},
},
required: ['key', 'op', 'value'],
},
},
},
required: ['filters'],
},
autoApply: true,
execute: async ({ filters }): Promise<ActionResult> => {
const baseQuery = deps.currentQuery.builder.queryData[0];
if (!baseQuery) {
throw new Error('No active query found in Traces Explorer.');
}
const tagItems = filters.map(aiFilterToTagFilterItem);
const newFilters = { items: tagItems, op: 'AND' };
const updatedBuilderQuery: IBuilderQuery = {
...baseQuery,
filters: newFilters,
filter: convertFiltersToExpression(newFilters),
};
deps.handleSetQueryData(0, updatedBuilderQuery);
deps.redirectWithQueryBuilderData(
replaceFirstQueryData(deps.currentQuery, updatedBuilderQuery),
);
return {
summary: `Query updated with ${filters.length} filter(s) and re-run.`,
};
},
getContext: (): Record<string, unknown> => ({
filters:
deps.currentQuery.builder.queryData[0]?.filters?.items?.map(
(f: TagFilterItem) => ({
key: f.key?.key,
op: f.op,
value: f.value,
}),
) ?? [],
}),
};
}
/**
* Append a single filter to the existing trace query and navigate to the
* updated URL.
*/
export function tracesAddFilterAction(
deps: FilterDeps,
): PageAction<AddFilterParams> {
return {
id: 'traces.addFilter',
description: 'Add a single filter to the current trace query and re-run',
parameters: {
type: 'object',
properties: {
key: {
type: 'string',
description: 'Attribute key, e.g. service.name, http.status_code',
},
op: {
type: 'string',
enum: [...FILTER_OP_ENUM],
},
value: {
type: 'string',
description: FILTER_VALUE_DESCRIPTION,
},
},
required: ['key', 'op', 'value'],
},
autoApply: true,
execute: async ({ key, op, value }): Promise<ActionResult> => {
const baseQuery = deps.currentQuery.builder.queryData[0];
if (!baseQuery) {
throw new Error('No active query found in Traces Explorer.');
}
const existing = baseQuery.filters?.items ?? [];
const newItem = aiFilterToTagFilterItem({ key, op, value });
const newFilters = { items: [...existing, newItem], op: 'AND' };
const updatedBuilderQuery: IBuilderQuery = {
...baseQuery,
filters: newFilters,
filter: convertFiltersToExpression(newFilters),
};
deps.handleSetQueryData(0, updatedBuilderQuery);
deps.redirectWithQueryBuilderData(
replaceFirstQueryData(deps.currentQuery, updatedBuilderQuery),
);
return { summary: `Filter added: ${key} ${op} "${value}". Query re-run.` };
},
};
}
/**
* Switch the traces explorer between list / timeseries / table / trace views.
*/
export function tracesChangeViewAction(deps: {
onChangeView: (view: TracesView) => void;
}): PageAction<ChangeViewParams> {
return {
id: 'traces.changeView',
description:
'Switch the Traces Explorer between list, timeseries, table, and trace views',
parameters: {
type: 'object',
properties: {
view: {
type: 'string',
enum: ['list', 'timeseries', 'table', 'trace'],
description: 'The panel view to switch to',
},
},
required: ['view'],
},
execute: async ({ view }): Promise<ActionResult> => {
deps.onChangeView(view);
return { summary: `Switched to the "${view}" view.` };
},
};
}
/**
* Save the current trace query as a named view (stub — wires to real API
* when available).
*/
export function tracesSaveViewAction(deps: {
onSaveView: (name: string) => Promise<void>;
}): PageAction<SaveViewParams> {
return {
id: 'traces.saveView',
description: 'Save the current trace query as a named view',
parameters: {
type: 'object',
properties: {
name: { type: 'string', description: 'Name for the saved view' },
},
required: ['name'],
},
execute: async ({ name }): Promise<ActionResult> => {
await deps.onSaveView(name);
return { summary: `View "${name}" saved.` };
},
};
}

View File

@@ -0,0 +1,132 @@
import {
ArrowUpToLine,
Atom,
Filter,
SquareMousePointer,
Terminal,
Binoculars,
} from '@signozhq/icons';
import { Button, Tooltip } from 'antd';
import cx from 'classnames';
import { ExplorerViews } from 'pages/LogsExplorer/utils';
import './ToolbarActions.styles.scss';
interface LeftToolbarActionsProps {
items: any;
selectedView: string;
onChangeSelectedView: (view: ExplorerViews) => void;
showFilter: boolean;
handleFilterVisibilityChange: () => void;
}
const activeTab = 'active-tab';
export default function LeftToolbarActions({
items,
selectedView,
onChangeSelectedView,
showFilter,
handleFilterVisibilityChange,
}: LeftToolbarActionsProps): JSX.Element {
const { clickhouse, list, timeseries, table, trace } = items;
return (
<div className="left-toolbar">
{!showFilter && (
<Tooltip title="Show Filters">
<Button onClick={handleFilterVisibilityChange} className="filter-btn">
<Filter size={12} />
<ArrowUpToLine size={12} style={{ transform: 'rotate(90deg)' }} />
</Button>
</Tooltip>
)}
<div className="left-toolbar-query-actions">
{list?.show && (
<Tooltip title="List View">
<Button
disabled={list.disabled}
className={cx(
'list-view-tab',
'explorer-view-option',
selectedView === list.key ? activeTab : '',
)}
onClick={(): void => onChangeSelectedView(list.key)}
>
<SquareMousePointer size={14} data-testid="search-view" />
List View
</Button>
</Tooltip>
)}
{trace?.show && (
<Tooltip title="Trace View">
<Button
disabled={trace.disabled}
className={cx(
'trace-view-tab',
'explorer-view-option',
selectedView === trace.key ? activeTab : '',
)}
onClick={(): void => onChangeSelectedView(trace.key)}
>
<SquareMousePointer size={14} data-testid="trace-view" />
Trace View
</Button>
</Tooltip>
)}
{timeseries?.show && (
<Tooltip title="Time Series">
<Button
disabled={timeseries.disabled}
className={cx(
'timeseries-view-tab',
'explorer-view-option',
selectedView === timeseries.key ? activeTab : '',
)}
onClick={(): void => onChangeSelectedView(timeseries.key)}
>
<Atom size={14} data-testid="query-builder-view" />
Time Series
</Button>
</Tooltip>
)}
{clickhouse?.show && (
<Tooltip title="Clickhouse">
<Button
disabled={clickhouse.disabled}
className={cx(
'clickhouse-view-tab',
'explorer-view-option',
selectedView === clickhouse.key ? activeTab : '',
)}
onClick={(): void => onChangeSelectedView(clickhouse.key)}
>
<Terminal size={14} data-testid="clickhouse-view" />
Clickhouse
</Button>
</Tooltip>
)}
{table?.show && (
<Tooltip title="Table">
<Button
disabled={table.disabled}
className={cx(
'table-view-tab',
'explorer-view-option',
selectedView === table.key ? activeTab : '',
)}
onClick={(): void => onChangeSelectedView(table.key)}
>
<Binoculars size={14} data-testid="query-builder-view-v2" />
Table
</Button>
</Tooltip>
)}
</div>
</div>
);
}

View File

@@ -0,0 +1,125 @@
.left-toolbar {
display: flex;
align-items: center;
.filter-btn {
display: flex;
align-items: center;
justify-content: center;
box-shadow: none;
height: 32px;
margin-right: 12px;
border: 1px solid var(--l1-border);
}
.left-toolbar-query-actions {
display: flex;
border-radius: 2px;
border: 1px solid var(--l1-border);
background: var(--l1-background);
flex-direction: row;
border-bottom: none;
margin-bottom: -1px;
.prom-ql-icon {
height: 14px;
width: 14px;
}
.explorer-view-option {
display: flex;
align-items: center;
justify-content: center;
flex-direction: row;
border: none;
padding: 9px;
box-shadow: none;
border-radius: 0px;
border-left: 1px solid var(--l1-border);
border-bottom: 1px solid var(--l1-border);
gap: 8px;
&.active-tab {
background-color: var(--primary-background);
border-bottom: 1px solid var(--primary-background);
color: var(--primary-foreground);
&:hover {
background-color: var(--primary-background) !important;
}
}
&:disabled {
background-color: var(--l3-background);
opacity: 0.6;
}
&:first-child {
border-left: 1px solid transparent;
}
&:hover {
background-color: transparent !important;
border-left: 1px solid transparent !important;
color: var(--l1-foreground);
}
}
}
.frequency-chart-view-controller {
display: flex;
align-items: center;
padding-left: 8px;
gap: 8px;
}
}
.right-toolbar {
display: flex;
align-items: center;
background-color: var(--bg-robin-600);
}
.right-actions {
display: flex;
align-items: center;
}
.loading-container {
display: flex;
gap: 8px;
align-items: center;
.loading-btn {
display: flex;
width: 32px;
height: 33px;
padding: 4px 10px;
justify-content: center;
align-items: center;
gap: 6px;
flex-shrink: 0;
border-radius: 2px;
background: var(--l3-background);
box-shadow: none;
border: none;
}
.cancel-run {
display: flex;
height: 33px;
padding: 4px 10px;
justify-content: center;
align-items: center;
gap: 6px;
flex: 1 0 0;
border-radius: 2px;
background: var(--danger-background);
border: none;
}
.cancel-run:hover {
background-color: var(--bg-cherry-400) !important;
color: var(--l1-foreground) !important;
}
}

1
go.mod
View File

@@ -57,7 +57,6 @@ require (
github.com/segmentio/analytics-go/v3 v3.2.1
github.com/sethvargo/go-password v0.2.0
github.com/smartystreets/goconvey v1.8.1
github.com/soheilhy/cmux v0.1.5
github.com/spf13/cobra v1.10.2
github.com/stretchr/testify v1.11.1
github.com/swaggest/jsonschema-go v0.3.78

3
go.sum
View File

@@ -1057,8 +1057,6 @@ github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1
github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA=
github.com/smartystreets/goconvey v1.8.1 h1:qGjIddxOk4grTu9JPOU31tVfq3cNdBlNa5sSznIX1xY=
github.com/smartystreets/goconvey v1.8.1/go.mod h1:+/u4qLyY6x1jReYOp7GOM2FSt8aP9CzCZL03bI28W60=
github.com/soheilhy/cmux v0.1.5 h1:jjzc5WVemNEDTLwv9tlmemhC73tI08BNOIGwBOo10Js=
github.com/soheilhy/cmux v0.1.5/go.mod h1:T7TcVDs9LWfQgPlPsdngu6I6QIoyIFZDDC6sNE1GqG0=
github.com/sourcegraph/conc v0.3.0 h1:OQTbbt6P72L20UqAkXXuLOj79LfEanQ+YQFNpLA9ySo=
github.com/sourcegraph/conc v0.3.0/go.mod h1:Sdozi7LEKbFPqYX2/J+iBAM6HpqSLTASQIKqDmF7Mt0=
github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
@@ -1493,7 +1491,6 @@ golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81R
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201031054903-ff519b6c9102/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201202161906-c7110b5ffcbb/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201209123823-ac852fbbde11/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20201224014010-6772e930b67b/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210119194325-5f4716e94777/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=

View File

@@ -1,10 +1,14 @@
package apiserver
import (
"github.com/SigNoz/signoz/pkg/factory"
"github.com/gorilla/mux"
)
type APIServer interface {
// APIServer is a long running service serving the SigNoz API.
factory.ServiceWithHealthy
// Returns the mux router for the API server. Primarily used for collecting OpenAPI operations.
Router() *mux.Router

View File

@@ -3,13 +3,16 @@ package apiserver
import (
"time"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
httpserver "github.com/SigNoz/signoz/pkg/http/server"
)
// Config holds the configuration for config.
type Config struct {
Timeout Timeout `mapstructure:"timeout"`
Logging Logging `mapstructure:"logging"`
httpserver.Config `mapstructure:",squash" yaml:",squash"`
Timeout Timeout `mapstructure:"timeout"`
Logging Logging `mapstructure:"logging"`
}
type Timeout struct {
@@ -32,6 +35,10 @@ func NewConfigFactory() factory.ConfigFactory {
func newConfig() factory.Config {
return &Config{
Config: httpserver.Config{
Address: "0.0.0.0:8080",
ReadTimeout: 60 * time.Second,
},
Timeout: Timeout{
Default: 60 * time.Second,
Max: 600 * time.Second,
@@ -52,5 +59,9 @@ func newConfig() factory.Config {
}
func (c Config) Validate() error {
if c.Address == "" {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "apiserver.address is required")
}
return nil
}

View File

@@ -8,11 +8,14 @@ import (
"github.com/SigNoz/signoz/pkg/config"
"github.com/SigNoz/signoz/pkg/config/envprovider"
"github.com/SigNoz/signoz/pkg/factory"
httpserver "github.com/SigNoz/signoz/pkg/http/server"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestNewWithEnvProvider(t *testing.T) {
t.Setenv("SIGNOZ_APISERVER_ADDRESS", "0.0.0.0:9090")
t.Setenv("SIGNOZ_APISERVER_READ__TIMEOUT", "80s")
t.Setenv("SIGNOZ_APISERVER_TIMEOUT_DEFAULT", "70s")
t.Setenv("SIGNOZ_APISERVER_TIMEOUT_MAX", "700s")
t.Setenv("SIGNOZ_APISERVER_TIMEOUT_EXCLUDED__ROUTES", "/excluded1,/excluded2")
@@ -38,6 +41,10 @@ func TestNewWithEnvProvider(t *testing.T) {
require.NoError(t, err)
expected := &Config{
Config: httpserver.Config{
Address: "0.0.0.0:9090",
ReadTimeout: 80 * time.Second,
},
Timeout: Timeout{
Default: 70 * time.Second,
Max: 700 * time.Second,

View File

@@ -2,9 +2,11 @@ package signozapiserver
import (
"context"
"net/http"
"github.com/SigNoz/signoz/pkg/alertmanager"
"github.com/SigNoz/signoz/pkg/apiserver"
"github.com/SigNoz/signoz/pkg/auditor"
"github.com/SigNoz/signoz/pkg/authz"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
@@ -12,6 +14,8 @@ import (
"github.com/SigNoz/signoz/pkg/global"
"github.com/SigNoz/signoz/pkg/http/handler"
"github.com/SigNoz/signoz/pkg/http/middleware"
httpserver "github.com/SigNoz/signoz/pkg/http/server"
"github.com/SigNoz/signoz/pkg/identn"
"github.com/SigNoz/signoz/pkg/licensing"
"github.com/SigNoz/signoz/pkg/modules/aiobservability"
"github.com/SigNoz/signoz/pkg/modules/authdomain"
@@ -37,18 +41,22 @@ import (
"github.com/SigNoz/signoz/pkg/prometheus"
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/ruler"
"github.com/SigNoz/signoz/pkg/sharder"
"github.com/SigNoz/signoz/pkg/statsreporter"
"github.com/SigNoz/signoz/pkg/subscription"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/web"
"github.com/SigNoz/signoz/pkg/zeus"
"github.com/gorilla/mux"
)
type provider struct {
config apiserver.Config
settings factory.ScopedProviderSettings
globalConfig global.Config
web web.Web
router *mux.Router
httpServer *httpserver.Server
healthyC chan struct{}
authzMiddleware *middleware.AuthZ
authzService authz.AuthZ
orgHandler organization.Handler
@@ -132,6 +140,11 @@ func NewFactory(
rulerHandler ruler.Handler,
statsHandler statsreporter.Handler,
savedViewHandler savedview.Handler,
globalConfig global.Config,
identNResolver identn.IdentNResolver,
sharder sharder.Sharder,
auditor auditor.Auditor,
web web.Web,
quickFilterModule quickfilter.Module,
quickFilterHandler quickfilter.Handler,
) factory.ProviderFactory[apiserver.APIServer, apiserver.Config] {
@@ -179,6 +192,11 @@ func NewFactory(
rulerHandler,
statsHandler,
savedViewHandler,
globalConfig,
identNResolver,
sharder,
auditor,
web,
quickFilterModule,
quickFilterHandler,
)
@@ -228,6 +246,11 @@ func newProvider(
rulerHandler ruler.Handler,
statsHandler statsreporter.Handler,
savedViewHandler savedview.Handler,
globalConfig global.Config,
identNResolver identn.IdentNResolver,
sharder sharder.Sharder,
auditor auditor.Auditor,
web web.Web,
quickFilterModule quickfilter.Module,
quickFilterHandler quickfilter.Handler,
) (apiserver.APIServer, error) {
@@ -235,9 +258,10 @@ func newProvider(
router := mux.NewRouter().UseEncodedPath()
provider := &provider{
config: config,
settings: settings,
globalConfig: globalConfig,
web: web,
router: router,
healthyC: make(chan struct{}),
orgHandler: orgHandler,
userHandler: userHandler,
authzService: authzService,
@@ -282,13 +306,68 @@ func newProvider(
provider.authzMiddleware = middleware.NewAuthZ(settings.Logger(), orgGetter, authzService)
router.Use(middleware.NewRecovery(settings.Logger()).Wrap)
router.Use(middleware.NewOtel("apiserver", providerSettings.MeterProvider, providerSettings.TracerProvider).Wrap)
router.Use(middleware.NewIdentN(identNResolver, sharder, settings.Logger()).Wrap)
router.Use(middleware.NewTimeout(settings.Logger(),
config.Timeout.ExcludedRoutes,
config.Timeout.Default,
config.Timeout.Max,
).Wrap)
router.Use(middleware.NewResource(settings.Logger()).Wrap)
router.Use(middleware.NewAudit(settings.Logger(), config.Logging.ExcludedRoutes, auditor).Wrap)
router.Use(middleware.NewComment().Wrap)
if err := provider.AddToRouter(router); err != nil {
return nil, err
}
httpHandler := middleware.NewCors().Wrap(router)
httpHandler = middleware.NewCompress().Wrap(httpHandler)
routePrefix := globalConfig.ExternalPath()
if routePrefix != "" {
prefixed := http.StripPrefix(routePrefix, httpHandler)
httpHandler = http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
switch req.URL.Path {
case "/api/v1/health", "/api/v2/healthz", "/api/v2/readyz", "/api/v2/livez":
router.ServeHTTP(w, req)
return
}
prefixed.ServeHTTP(w, req)
})
}
httpServer, err := httpserver.New(settings.Logger(), config.Config, httpHandler)
if err != nil {
return nil, err
}
provider.httpServer = httpServer
return provider, nil
}
func (provider *provider) Start(ctx context.Context) error {
// Mount the web routes last so the catch-all prefix does not shadow API
// routes registered on the router after construction.
if err := provider.web.AddToRouter(provider.router); err != nil {
return err
}
close(provider.healthyC)
return provider.httpServer.Start(ctx)
}
func (provider *provider) Stop(ctx context.Context) error {
return provider.httpServer.Stop(ctx)
}
func (provider *provider) Healthy() <-chan struct{} {
return provider.healthyC
}
func (provider *provider) Router() *mux.Router {
return provider.router
}

View File

@@ -78,6 +78,40 @@ func NewRegistry(ctx context.Context, logger *slog.Logger, services ...NamedServ
}, nil
}
// Add registers additional services into the registry. It must be called before Start.
func (registry *Registry) Add(ctx context.Context, services ...NamedService) error {
added := make([]*serviceWithState, 0, len(services))
for _, s := range services {
if _, ok := registry.servicesByName[s.Name()]; ok {
return errors.Newf(errors.TypeInvalidInput, ErrCodeInvalidRegistry, "cannot add service, duplicate service name %q", s.Name())
}
added = append(added, newServiceWithState(s))
}
for _, ss := range added {
registry.services = append(registry.services, ss)
registry.servicesByName[ss.service.Name()] = ss
}
for _, ss := range added {
for _, dep := range ss.service.DependsOn() {
if dep == ss.service.Name() {
registry.logger.ErrorContext(ctx, "ignoring self-dependency", slog.Any("service", ss.service.Name()))
continue
}
if _, ok := registry.servicesByName[dep]; !ok {
registry.logger.ErrorContext(ctx, "ignoring unknown dependency", slog.Any("service", ss.service.Name()), slog.Any("dependency", dep))
continue
}
ss.dependsOn = append(ss.dependsOn, dep)
}
}
return detectCyclicDeps(registry.services)
}
func (registry *Registry) Start(ctx context.Context) {
for _, ss := range registry.services {
go func(ss *serviceWithState) {

View File

@@ -342,3 +342,61 @@ func TestDependsOnCycleReturnsError(t *testing.T) {
assert.Error(t, err)
assert.Contains(t, err.Error(), "dependency cycles detected")
}
func TestRegistryAdd(t *testing.T) {
s1 := newTestService(t)
s2 := newTestService(t)
registry, err := NewRegistry(context.Background(), slog.New(slog.DiscardHandler), NewNamedService(MustNewName("s1"), s1))
require.NoError(t, err)
require.NoError(t, registry.Add(context.Background(), NewNamedService(MustNewName("s2"), s2)))
ctx := context.Background()
registry.Start(ctx)
require.NoError(t, registry.AwaitHealthy(ctx))
byState := registry.ServicesByState()
assert.Len(t, byState[StateRunning], 2)
assert.True(t, registry.IsHealthy())
assert.NoError(t, registry.Stop(ctx))
}
func TestRegistryAddDuplicateReturnsError(t *testing.T) {
s1 := newTestService(t)
registry, err := NewRegistry(context.Background(), slog.New(slog.DiscardHandler), NewNamedService(MustNewName("s1"), s1))
require.NoError(t, err)
err = registry.Add(context.Background(), NewNamedService(MustNewName("s1"), newTestService(t)))
assert.Error(t, err)
assert.Contains(t, err.Error(), "duplicate service name")
}
func TestRegistryAddWithDependency(t *testing.T) {
s1 := newHealthyTestService(t)
s2 := newTestService(t)
registry, err := NewRegistry(context.Background(), slog.New(slog.DiscardHandler), NewNamedService(MustNewName("s1"), s1))
require.NoError(t, err)
// s2 depends on the already registered s1.
require.NoError(t, registry.Add(context.Background(), NewNamedService(MustNewName("s2"), s2, MustNewName("s1"))))
ctx := context.Background()
registry.Start(ctx)
// s2 stays in STARTING until s1 is healthy.
require.Eventually(t, func() bool {
byState := registry.ServicesByState()
return len(byState[StateStarting]) == 2
}, time.Second, time.Millisecond)
close(s1.healthyC)
require.NoError(t, registry.AwaitHealthy(ctx))
assert.True(t, registry.IsHealthy())
assert.NoError(t, registry.Stop(ctx))
}

View File

@@ -0,0 +1,17 @@
package middleware
import (
"net/http"
gorillahandlers "github.com/gorilla/handlers"
)
type Compress struct{}
func NewCompress() *Compress {
return &Compress{}
}
func (middleware *Compress) Wrap(next http.Handler) http.Handler {
return gorillahandlers.CompressHandler(next)
}

View File

@@ -0,0 +1,25 @@
package middleware
import (
"net/http"
"github.com/rs/cors"
)
type Cors struct {
cors *cors.Cors
}
func NewCors() *Cors {
return &Cors{
cors: cors.New(cors.Options{
AllowedOrigins: []string{"*"},
AllowedMethods: []string{"GET", "DELETE", "POST", "PUT", "PATCH", "OPTIONS"},
AllowedHeaders: []string{"Accept", "Authorization", "Content-Type", "cache-control"},
}),
}
}
func (middleware *Cors) Wrap(next http.Handler) http.Handler {
return middleware.cors.Handler(next)
}

View File

@@ -0,0 +1,43 @@
package middleware
import (
"net/http"
"slices"
"github.com/gorilla/mux"
"go.opentelemetry.io/contrib/instrumentation/github.com/gorilla/mux/otelmux"
"go.opentelemetry.io/otel/metric"
"go.opentelemetry.io/otel/propagation"
"go.opentelemetry.io/otel/trace"
)
// defaultExcludedRoutes are the health endpoints kept out of tracing/metrics to
// avoid drowning telemetry in probe traffic.
var defaultExcludedRoutes = []string{
"/api/v1/health",
"/api/v2/healthz",
"/api/v2/readyz",
"/api/v2/livez",
}
type Otel struct {
wrap mux.MiddlewareFunc
}
func NewOtel(service string, meterProvider metric.MeterProvider, tracerProvider trace.TracerProvider) *Otel {
return &Otel{
wrap: otelmux.Middleware(
service,
otelmux.WithMeterProvider(meterProvider),
otelmux.WithTracerProvider(tracerProvider),
otelmux.WithPropagators(propagation.NewCompositeTextMapPropagator(propagation.Baggage{}, propagation.TraceContext{})),
otelmux.WithFilter(func(r *http.Request) bool {
return !slices.Contains(defaultExcludedRoutes, r.URL.Path)
}),
),
}
}
func (middleware *Otel) Wrap(next http.Handler) http.Handler {
return middleware.wrap(next)
}

View File

@@ -1,9 +1,18 @@
package server
import "time"
// Config holds the configuration for http.
type Config struct {
//Address specifies the TCP address for the server to listen on, in the form "host:port".
// If empty, ":http" (port 80) is used. The service names are defined in RFC 6335 and assigned by IANA.
// See net.Dial for details of the address format.
Address string `mapstructure:"address"`
// ReadTimeout bounds reading an entire request, including the body. Zero means no timeout.
ReadTimeout time.Duration `mapstructure:"read_timeout"`
// WriteTimeout bounds writing the response. Zero means no timeout, required for
// streaming endpoints that hold the connection open.
WriteTimeout time.Duration `mapstructure:"write_timeout"`
}

View File

@@ -31,8 +31,8 @@ func New(logger *slog.Logger, cfg Config, handler http.Handler) (*Server, error)
srv := &http.Server{
Addr: cfg.Address,
Handler: handler,
ReadTimeout: 10 * time.Second,
WriteTimeout: 10 * time.Second,
ReadTimeout: cfg.ReadTimeout,
WriteTimeout: cfg.WriteTimeout,
MaxHeaderBytes: 1 << 20,
}

View File

@@ -7,7 +7,6 @@ import (
"github.com/SigNoz/signoz/pkg/modules/dashboard"
"github.com/SigNoz/signoz/pkg/modules/organization"
"github.com/SigNoz/signoz/pkg/modules/quickfilter"
"github.com/SigNoz/signoz/pkg/modules/spanmapper"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/valuer"
)
@@ -17,11 +16,10 @@ type setter struct {
alertmanager alertmanager.Alertmanager
quickfilter quickfilter.Module
dashboard dashboard.Module
spanMapper spanmapper.Module
}
func NewSetter(store types.OrganizationStore, alertmanager alertmanager.Alertmanager, quickfilter quickfilter.Module, dashboard dashboard.Module, spanMapper spanmapper.Module) organization.Setter {
return &setter{store: store, alertmanager: alertmanager, quickfilter: quickfilter, dashboard: dashboard, spanMapper: spanMapper}
func NewSetter(store types.OrganizationStore, alertmanager alertmanager.Alertmanager, quickfilter quickfilter.Module, dashboard dashboard.Module) organization.Setter {
return &setter{store: store, alertmanager: alertmanager, quickfilter: quickfilter, dashboard: dashboard}
}
func (module *setter) Create(ctx context.Context, organization *types.Organization, createManagedRoles func(context.Context, valuer.UUID) error) error {
@@ -45,10 +43,6 @@ func (module *setter) Create(ctx context.Context, organization *types.Organizati
return err
}
if err := module.spanMapper.ReconcileSystemGroups(ctx, organization.ID); err != nil {
return err
}
return nil
}

View File

@@ -1,46 +0,0 @@
package implspanmapper
import (
"embed"
"io/fs"
"path"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types/spantypes"
)
const definitionsRoot = "fs/definitions"
//go:embed fs/definitions/*.json
var definitionFiles embed.FS
// NewSystemGroupRegistry parses every embedded definition. Definitions are
// build-time assets validated by a test, so a failure here means the binary
// shipped broken JSON.
func NewSystemGroupRegistry() (spantypes.SpanMapperGroupRegistry, error) {
entries, err := fs.ReadDir(definitionFiles, definitionsRoot)
if err != nil {
return spantypes.SpanMapperGroupRegistry{}, errors.WrapInternalf(err, errors.CodeInternal, "couldn't read span mapper group definitions")
}
definitions := make([]spantypes.SpanMapperGroupDefinition, 0, len(entries))
for _, entry := range entries {
if entry.IsDir() {
continue
}
file := path.Join(definitionsRoot, entry.Name())
raw, err := definitionFiles.ReadFile(file)
if err != nil {
return spantypes.SpanMapperGroupRegistry{}, errors.WrapInternalf(err, errors.CodeInternal, "couldn't read %s", file)
}
definition, err := spantypes.NewSpanMapperGroupDefinition(raw)
if err != nil {
return spantypes.SpanMapperGroupRegistry{}, errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "couldn't parse %s", file)
}
definitions = append(definitions, definition)
}
return spantypes.NewSpanMapperGroupRegistry(definitions)
}

View File

@@ -1,79 +0,0 @@
{
"version": 1,
"definition": {
"name": "agent",
"condition": {
"attributes": [
{
"value": "agent"
}
],
"resource": []
},
"enabled": true,
"mappers": [
{
"name": "gen_ai.agent.name",
"fieldContext": "attribute",
"config": {
"sources": [
{
"key": "agent.name",
"context": "attribute",
"operation": "copy",
"priority": 20
},
{
"key": "agent_name",
"context": "attribute",
"operation": "copy",
"priority": 10
}
]
}
},
{
"name": "gen_ai.agent.id",
"fieldContext": "attribute",
"config": {
"sources": [
{
"key": "agent.id",
"context": "attribute",
"operation": "copy",
"priority": 10
}
]
}
},
{
"name": "gen_ai.agent.description",
"fieldContext": "attribute",
"config": {
"sources": [
{
"key": "agent.description",
"context": "attribute",
"operation": "copy",
"priority": 10
}
]
}
},
{
"name": "gen_ai.output.messages",
"fieldContext": "attribute",
"config": {
"sources": [
{
"key": "final_result",
"context": "attribute",
"operation": "copy",
"priority": 10
}
]
}
}
]
}
}

View File

@@ -1,347 +0,0 @@
{
"version": 1,
"definition": {
"name": "llm",
"condition": {
"attributes": [
{
"value": "model"
}
],
"resource": []
},
"enabled": true,
"mappers": [
{
"name": "gen_ai.request.model",
"fieldContext": "attribute",
"config": {
"sources": [
{
"key": "llm.model_name",
"context": "attribute",
"operation": "copy",
"priority": 60
},
{
"key": "llm.request.model",
"context": "attribute",
"operation": "copy",
"priority": 50
},
{
"key": "ai.model.id",
"context": "attribute",
"operation": "copy",
"priority": 40
},
{
"key": "langfuse.observation.model.name",
"context": "attribute",
"operation": "copy",
"priority": 30
},
{
"key": "embedding.model_name",
"context": "attribute",
"operation": "copy",
"priority": 20
},
{
"key": "model",
"context": "attribute",
"operation": "copy",
"priority": 10
}
]
}
},
{
"name": "gen_ai.response.model",
"fieldContext": "attribute",
"config": {
"sources": [
{
"key": "llm.response.model",
"context": "attribute",
"operation": "copy",
"priority": 20
},
{
"key": "ai.response.model",
"context": "attribute",
"operation": "copy",
"priority": 10
}
]
}
},
{
"name": "gen_ai.provider.name",
"fieldContext": "attribute",
"config": {
"sources": [
{
"key": "gen_ai.system",
"context": "attribute",
"operation": "copy",
"priority": 50
},
{
"key": "llm.vendor",
"context": "attribute",
"operation": "copy",
"priority": 40
},
{
"key": "llm.provider",
"context": "attribute",
"operation": "copy",
"priority": 30
},
{
"key": "llm.system",
"context": "attribute",
"operation": "copy",
"priority": 20
},
{
"key": "ai.model.provider",
"context": "attribute",
"operation": "copy",
"priority": 10
}
]
}
},
{
"name": "gen_ai.operation.name",
"fieldContext": "attribute",
"config": {
"sources": [
{
"key": "llm.request.type",
"context": "attribute",
"operation": "copy",
"priority": 10
}
]
}
},
{
"name": "gen_ai.usage.input_tokens",
"fieldContext": "attribute",
"config": {
"sources": [
{
"key": "gen_ai.usage.prompt_tokens",
"context": "attribute",
"operation": "copy",
"priority": 50
},
{
"key": "llm.usage.prompt_tokens",
"context": "attribute",
"operation": "copy",
"priority": 40
},
{
"key": "llm.token_count.prompt",
"context": "attribute",
"operation": "copy",
"priority": 30
},
{
"key": "ai.usage.inputTokens",
"context": "attribute",
"operation": "copy",
"priority": 20
},
{
"key": "ai.usage.promptTokens",
"context": "attribute",
"operation": "copy",
"priority": 10
}
]
}
},
{
"name": "gen_ai.usage.output_tokens",
"fieldContext": "attribute",
"config": {
"sources": [
{
"key": "gen_ai.usage.completion_tokens",
"context": "attribute",
"operation": "copy",
"priority": 50
},
{
"key": "llm.usage.completion_tokens",
"context": "attribute",
"operation": "copy",
"priority": 40
},
{
"key": "llm.token_count.completion",
"context": "attribute",
"operation": "copy",
"priority": 30
},
{
"key": "ai.usage.outputTokens",
"context": "attribute",
"operation": "copy",
"priority": 20
},
{
"key": "ai.usage.completionTokens",
"context": "attribute",
"operation": "copy",
"priority": 10
}
]
}
},
{
"name": "gen_ai.usage.cache_read.input_tokens",
"fieldContext": "attribute",
"config": {
"sources": [
{
"key": "gen_ai.usage.cache_read_input_tokens",
"context": "attribute",
"operation": "copy",
"priority": 30
},
{
"key": "llm.token_count.prompt_details.cache_read",
"context": "attribute",
"operation": "copy",
"priority": 20
},
{
"key": "ai.usage.cachedInputTokens",
"context": "attribute",
"operation": "copy",
"priority": 10
}
]
}
},
{
"name": "gen_ai.usage.cache_creation.input_tokens",
"fieldContext": "attribute",
"config": {
"sources": [
{
"key": "gen_ai.usage.cache_write.input_tokens",
"context": "attribute",
"operation": "copy",
"priority": 30
},
{
"key": "gen_ai.usage.cache_creation_input_tokens",
"context": "attribute",
"operation": "copy",
"priority": 20
},
{
"key": "llm.token_count.prompt_details.cache_write",
"context": "attribute",
"operation": "copy",
"priority": 10
}
]
}
},
{
"name": "gen_ai.input.messages",
"fieldContext": "attribute",
"config": {
"sources": [
{
"key": "gen_ai.prompt",
"context": "attribute",
"operation": "copy",
"priority": 30
},
{
"key": "ai.prompt.messages",
"context": "attribute",
"operation": "copy",
"priority": 20
},
{
"key": "input.value",
"context": "attribute",
"operation": "copy",
"priority": 10
}
]
}
},
{
"name": "gen_ai.output.messages",
"fieldContext": "attribute",
"config": {
"sources": [
{
"key": "gen_ai.completion",
"context": "attribute",
"operation": "copy",
"priority": 30
},
{
"key": "ai.response.text",
"context": "attribute",
"operation": "copy",
"priority": 20
},
{
"key": "output.value",
"context": "attribute",
"operation": "copy",
"priority": 10
}
]
}
},
{
"name": "gen_ai.conversation.id",
"fieldContext": "attribute",
"config": {
"sources": [
{
"key": "session.id",
"context": "attribute",
"operation": "copy",
"priority": 20
},
{
"key": "langfuse.session.id",
"context": "attribute",
"operation": "copy",
"priority": 10
}
]
}
},
{
"name": "gen_ai.response.finish_reason",
"fieldContext": "attribute",
"config": {
"sources": [
{
"key": "ai.response.finishReason",
"context": "attribute",
"operation": "copy",
"priority": 10
}
]
}
}
]
}
}

View File

@@ -1,135 +0,0 @@
{
"version": 1,
"definition": {
"name": "tool",
"condition": {
"attributes": [
{
"value": "tool"
}
],
"resource": []
},
"enabled": true,
"mappers": [
{
"name": "gen_ai.tool.name",
"fieldContext": "attribute",
"config": {
"sources": [
{
"key": "tool.name",
"context": "attribute",
"operation": "copy",
"priority": 20
},
{
"key": "ai.toolCall.name",
"context": "attribute",
"operation": "copy",
"priority": 10
}
]
}
},
{
"name": "gen_ai.tool.call.id",
"fieldContext": "attribute",
"config": {
"sources": [
{
"key": "tool.id",
"context": "attribute",
"operation": "copy",
"priority": 20
},
{
"key": "ai.toolCall.id",
"context": "attribute",
"operation": "copy",
"priority": 10
}
]
}
},
{
"name": "gen_ai.tool.description",
"fieldContext": "attribute",
"config": {
"sources": [
{
"key": "tool.description",
"context": "attribute",
"operation": "copy",
"priority": 10
}
]
}
},
{
"name": "gen_ai.tool.call.arguments",
"fieldContext": "attribute",
"config": {
"sources": [
{
"key": "ai.toolCall.args",
"context": "attribute",
"operation": "copy",
"priority": 40
},
{
"key": "traceloop.entity.input",
"context": "attribute",
"operation": "copy",
"priority": 30
},
{
"key": "gcp.vertex.agent.tool_call_args",
"context": "attribute",
"operation": "copy",
"priority": 20
},
{
"key": "input.value",
"context": "attribute",
"operation": "copy",
"priority": 10
}
]
}
},
{
"name": "gen_ai.tool.call.result",
"fieldContext": "attribute",
"config": {
"sources": [
{
"key": "ai.toolCall.result",
"context": "attribute",
"operation": "copy",
"priority": 40
},
{
"key": "traceloop.entity.output",
"context": "attribute",
"operation": "copy",
"priority": 30
},
{
"key": "gcp.vertex.agent.tool_response",
"context": "attribute",
"operation": "copy",
"priority": 20
},
{
"key": "output.value",
"context": "attribute",
"operation": "copy",
"priority": 10
}
]
}
}
]
}
}

View File

@@ -69,10 +69,6 @@ func (h *handler) CreateGroup(rw http.ResponseWriter, r *http.Request) {
render.Error(rw, err)
return
}
if err := req.Validate(); err != nil {
render.Error(rw, err)
return
}
group := spantypes.NewSpanMapperGroup(orgID, claims.Email, req)
@@ -195,10 +191,6 @@ func (h *handler) CreateMapper(rw http.ResponseWriter, r *http.Request) {
render.Error(rw, err)
return
}
if err := req.Validate(); err != nil {
render.Error(rw, err)
return
}
mapper := spantypes.NewSpanMapper(groupID, claims.Email, req)
if err := h.module.CreateMapper(ctx, orgID, groupID, mapper); err != nil {

View File

@@ -5,7 +5,6 @@ import (
"encoding/json"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/modules/spanmapper"
"github.com/SigNoz/signoz/pkg/query-service/agentConf"
@@ -15,24 +14,13 @@ import (
"github.com/SigNoz/signoz/pkg/valuer"
)
// maxTestSpans bounds the input size: every test request boots a full
// in-memory collector pipeline and is reachable with viewer access.
const maxTestSpans = 100
type module struct {
store spantypes.SpanMapperStore
flagger flagger.Flagger
registry spantypes.SpanMapperGroupRegistry
settings factory.ScopedProviderSettings
store spantypes.SpanMapperStore
flagger flagger.Flagger
}
func NewModule(store spantypes.SpanMapperStore, flagger flagger.Flagger, registry spantypes.SpanMapperGroupRegistry, providerSettings factory.ProviderSettings) spanmapper.Module {
return &module{
store: store,
flagger: flagger,
registry: registry,
settings: factory.NewScopedProviderSettings(providerSettings, "github.com/SigNoz/signoz/pkg/modules/spanmapper/implspanmapper"),
}
func NewModule(store spantypes.SpanMapperStore, flagger flagger.Flagger) spanmapper.Module {
return &module{store: store, flagger: flagger}
}
func (module *module) ListGroups(ctx context.Context, orgID valuer.UUID, q *spantypes.ListSpanMapperGroupsQuery) ([]*spantypes.SpanMapperGroup, error) {
@@ -44,9 +32,6 @@ func (module *module) GetGroup(ctx context.Context, orgID, id valuer.UUID) (*spa
}
func (module *module) CreateGroup(ctx context.Context, orgID valuer.UUID, group *spantypes.SpanMapperGroup) error {
if module.registry.IsReserved(group.Name) {
return errors.Newf(errors.TypeInvalidInput, spantypes.ErrCodeMappingGroupNameReserved, "group name %q is reserved for a default group", group.Name)
}
return module.store.CreateGroup(ctx, group)
}
@@ -55,14 +40,10 @@ func (module *module) UpdateGroup(ctx context.Context, orgID, id valuer.UUID, na
if err != nil {
return err
}
if name != nil && *name != group.Name && module.registry.IsReserved(*name) {
return errors.Newf(errors.TypeInvalidInput, spantypes.ErrCodeMappingGroupNameReserved, "group name %q is reserved for a default group", *name)
}
if err := group.Update(name, condition, enabled, updatedBy); err != nil {
return err
}
group.Update(name, condition, enabled, updatedBy)
if err := module.store.UpdateGroup(ctx, group); err != nil {
err = module.store.UpdateGroup(ctx, group)
if err != nil {
return err
}
agentConf.NotifyConfigUpdate(ctx)
@@ -70,16 +51,10 @@ func (module *module) UpdateGroup(ctx context.Context, orgID, id valuer.UUID, na
}
func (module *module) DeleteGroup(ctx context.Context, orgID, id valuer.UUID) error {
group, err := module.store.GetGroup(ctx, orgID, id)
err := module.store.DeleteGroup(ctx, orgID, id)
if err != nil {
return err
}
if err := group.ErrIfNotDeletable(); err != nil {
return err
}
if err := module.store.DeleteGroup(ctx, orgID, id); err != nil {
return err
}
agentConf.NotifyConfigUpdate(ctx)
return nil
}
@@ -106,13 +81,14 @@ func (module *module) CreateMapper(ctx context.Context, orgID, groupID valuer.UU
}
func (module *module) UpdateMapper(ctx context.Context, orgID, groupID, id valuer.UUID, fieldContext spantypes.FieldContext, config *spantypes.SpanMapperConfig, enabled *bool, updatedBy string) error {
if _, err := module.store.GetGroup(ctx, orgID, groupID); err != nil {
return err
}
mapper, err := module.store.GetMapper(ctx, orgID, groupID, id)
if err != nil {
return err
}
if err := mapper.Update(fieldContext, config, enabled, updatedBy); err != nil {
return err
}
mapper.Update(fieldContext, config, enabled, updatedBy)
err = module.store.UpdateMapper(ctx, mapper)
if err != nil {
return err
@@ -122,14 +98,7 @@ func (module *module) UpdateMapper(ctx context.Context, orgID, groupID, id value
}
func (module *module) DeleteMapper(ctx context.Context, orgID, groupID, id valuer.UUID) error {
mapper, err := module.store.GetMapper(ctx, orgID, groupID, id)
if err != nil {
return err
}
if err := mapper.ErrIfNotDeletable(); err != nil {
return err
}
err = module.store.DeleteMapper(ctx, orgID, groupID, id)
err := module.store.DeleteMapper(ctx, orgID, groupID, id)
if err != nil {
return err
}
@@ -137,6 +106,10 @@ func (module *module) DeleteMapper(ctx context.Context, orgID, groupID, id value
return nil
}
// maxTestSpans bounds the input size: every test request boots a full
// in-memory collector pipeline and is reachable with viewer access.
const maxTestSpans = 100
func (module *module) TestMappers(ctx context.Context, orgID valuer.UUID, spans []spantypes.SpanMapperTestSpan, groups []*spantypes.SpanMapperGroupWithMappers) ([]spantypes.SpanMapperTestSpan, []string, error) {
if len(spans) == 0 {
return nil, nil, errors.New(errors.TypeInvalidInput, spantypes.ErrCodeMappingInvalidInput, "'spans' must contain at least one span")
@@ -157,6 +130,37 @@ func (module *module) TestMappers(ctx context.Context, orgID valuer.UUID, spans
return out, collectorLogs, nil
}
// backfillMappers loads saved mappers for any enabled group whose Mappers is
// nil. Disabled groups are skipped: the simulation filters them out anyway,
// so there is no point loading their mappers or failing on their names.
func (module *module) backfillMappers(ctx context.Context, orgID valuer.UUID, groups []*spantypes.SpanMapperGroupWithMappers) ([]*spantypes.SpanMapperGroupWithMappers, error) {
savedGroups, err := module.store.ListGroups(ctx, orgID, nil)
if err != nil {
return nil, err
}
savedByName := make(map[string]*spantypes.SpanMapperGroup, len(savedGroups))
for _, g := range savedGroups {
savedByName[g.Name] = g
}
// For each group in the request, if Mappers is nil, load the saved mappers for that group name.
for _, g := range groups {
if g.Mappers != nil || !g.Group.Enabled {
continue
}
saved, ok := savedByName[g.Group.Name]
if !ok {
return nil, errors.Newf(errors.TypeNotFound, spantypes.ErrCodeMappingGroupNotFound, "no saved group named %q to load mappers from; send 'mappers' for new or edited groups", g.Group.Name)
}
loaded, err := module.store.ListMappers(ctx, orgID, saved.ID)
if err != nil {
return nil, err
}
g.Mappers = loaded
}
return groups, nil
}
func (module *module) AgentFeatureType() agentConf.AgentFeatureType {
return spantypes.SpanAttrMappingFeatureType
}
@@ -192,37 +196,6 @@ func (module *module) RecommendAgentConfig(orgID valuer.UUID, currentConfYaml []
return updatedConf, string(serialized), nil
}
// backfillMappers loads saved mappers for any enabled group whose Mappers is
// nil. Disabled groups are skipped: the simulation filters them out anyway,
// so there is no point loading their mappers or failing on their names.
func (module *module) backfillMappers(ctx context.Context, orgID valuer.UUID, groups []*spantypes.SpanMapperGroupWithMappers) ([]*spantypes.SpanMapperGroupWithMappers, error) {
savedGroups, err := module.store.ListGroups(ctx, orgID, nil)
if err != nil {
return nil, err
}
savedByName := make(map[string]*spantypes.SpanMapperGroup, len(savedGroups))
for _, g := range savedGroups {
savedByName[g.Name] = g
}
// For each group in the request, if Mappers is nil, load the saved mappers for that group name.
for _, g := range groups {
if g.Mappers != nil || !g.Group.Enabled {
continue
}
saved, ok := savedByName[g.Group.Name]
if !ok {
return nil, errors.Newf(errors.TypeNotFound, spantypes.ErrCodeMappingGroupNotFound, "no saved group named %q to load mappers from; send 'mappers' for new or edited groups", g.Group.Name)
}
loaded, err := module.store.ListMappers(ctx, orgID, saved.ID)
if err != nil {
return nil, err
}
g.Mappers = loaded
}
return groups, nil
}
// listEnabledGroupsWithMappers returns groups with their mappers.
func (module *module) listEnabledGroupsWithMappers(ctx context.Context, orgID valuer.UUID) ([]*spantypes.SpanMapperGroupWithMappers, error) {
enabled := true

View File

@@ -1,228 +0,0 @@
package implspanmapper
import (
"context"
"path/filepath"
"strconv"
"testing"
"time"
"github.com/SigNoz/signoz/pkg/factory/factorytest"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/sqlstore/sqlitesqlstore"
"github.com/SigNoz/signoz/pkg/types/spantypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
const testUser = "user@signoz.io"
func newTestSQLStore(t *testing.T) sqlstore.SQLStore {
t.Helper()
store, err := sqlitesqlstore.New(context.Background(), factorytest.NewSettings(), sqlstore.Config{
Provider: "sqlite",
Connection: sqlstore.ConnectionConfig{MaxOpenConns: 10},
Sqlite: sqlstore.SqliteConfig{
Path: filepath.Join(t.TempDir(), "test.db"),
Mode: "wal",
BusyTimeout: 5 * time.Second,
TransactionMode: "deferred",
},
})
require.NoError(t, err)
for _, model := range []any{
(*spantypes.StorableSpanMapperGroup)(nil),
(*spantypes.StorableSpanMapper)(nil),
} {
_, err := store.BunDB().NewCreateTable().Model(model).IfNotExists().Exec(context.Background())
require.NoError(t, err)
}
_, err = store.BunDB().Exec(`CREATE UNIQUE INDEX IF NOT EXISTS uq_span_mapper_group_org_name ON span_mapper_group (org_id, name)`)
require.NoError(t, err)
_, err = store.BunDB().Exec(`CREATE UNIQUE INDEX IF NOT EXISTS uq_span_mapper_group_name ON span_mapper (group_id, name)`)
require.NoError(t, err)
return store
}
func newTestModule(t *testing.T, sqlStore sqlstore.SQLStore, definitions ...spantypes.SpanMapperGroupDefinition) *module {
t.Helper()
registry, err := spantypes.NewSpanMapperGroupRegistry(definitions)
require.NoError(t, err)
return NewModule(NewStore(sqlStore), nil, registry, factorytest.NewSettings()).(*module)
}
func newTestDefinition(t *testing.T, version int, body string) spantypes.SpanMapperGroupDefinition {
t.Helper()
definition, err := spantypes.NewSpanMapperGroupDefinition([]byte(`{"version": ` + strconv.Itoa(version) + `, "definition": ` + body + `}`))
require.NoError(t, err)
return definition
}
// llmV1 ships two mappers; llmV2 renames a source, adds a mapper and drops one.
const llmV1 = `{
"name": "llm",
"condition": {"attributes": [{"value": "model"}], "resource": []},
"enabled": true,
"mappers": [
{"name": "gen_ai.request.model", "fieldContext": "attribute", "config": {"sources": [
{"key": "llm.model_name", "context": "attribute", "operation": "copy", "priority": 20},
{"key": "ai.model.id", "context": "attribute", "operation": "copy", "priority": 10}
]}},
{"name": "gen_ai.input.messages", "fieldContext": "attribute", "config": {"sources": [
{"key": "gen_ai.prompt", "context": "attribute", "operation": "copy", "priority": 10}
]}}
]
}`
const llmV2 = `{
"name": "llm",
"condition": {"attributes": [{"value": "model"}, {"value": "llm."}], "resource": []},
"enabled": true,
"mappers": [
{"name": "gen_ai.request.model", "fieldContext": "attribute", "config": {"sources": [
{"key": "llm.model_name", "context": "attribute", "operation": "copy", "priority": 20},
{"key": "langfuse.observation.model.name", "context": "attribute", "operation": "copy", "priority": 10}
]}},
{"name": "gen_ai.provider.name", "fieldContext": "attribute", "config": {"sources": [
{"key": "llm.vendor", "context": "attribute", "operation": "copy", "priority": 10}
]}}
]
}`
func findMapper(t *testing.T, mappers []*spantypes.SpanMapper, name string) *spantypes.SpanMapper {
t.Helper()
for _, m := range mappers {
if m.Name == name {
return m
}
}
require.Failf(t, "mapper not found", "no mapper named %q", name)
return nil
}
func findSource(t *testing.T, sources []spantypes.SpanMapperSource, key string, origin spantypes.SpanMapperOrigin) spantypes.SpanMapperSource {
t.Helper()
for _, s := range sources {
if s.Key == key && s.Origin == origin {
return s
}
}
require.Failf(t, "source not found", "no %s source with key %q", origin.StringValue(), key)
return spantypes.SpanMapperSource{}
}
func TestReconcileUpgradeKeepsTogglesAndUserItems(t *testing.T) {
ctx := context.Background()
orgID := valuer.GenerateUUID()
sqlStore := newTestSQLStore(t)
v1 := newTestModule(t, sqlStore, newTestDefinition(t, 1, llmV1))
require.NoError(t, v1.ReconcileSystemGroups(ctx, orgID))
group, err := v1.store.GetGroupByName(ctx, orgID, "llm")
require.NoError(t, err)
mappers, err := v1.ListMappers(ctx, orgID, group.ID)
require.NoError(t, err)
// Switch the shipped substring off and add a user one.
off := false
require.NoError(t, v1.UpdateGroup(ctx, orgID, group.ID, nil, &spantypes.SpanMapperGroupCondition{
Attributes: []spantypes.SpanMapperGroupConditionKey{
{Value: "model", Enabled: false, Origin: spantypes.SpanMapperOriginSystem},
{Value: "gen_ai.request.model", Enabled: true, Origin: spantypes.SpanMapperOriginUser},
},
Resource: []spantypes.SpanMapperGroupConditionKey{},
}, &off, testUser))
// Switch a shipped source off, add a user override, and switch the mapper off.
model := findMapper(t, mappers, "gen_ai.request.model")
require.NoError(t, v1.UpdateMapper(ctx, orgID, group.ID, model.ID, spantypes.FieldContext{}, &spantypes.SpanMapperConfig{Sources: []spantypes.SpanMapperSource{
{Key: "llm.model_name", Context: spantypes.FieldContextSpanAttribute, Operation: spantypes.SpanMapperOperationCopy, Priority: 20, Enabled: false, Origin: spantypes.SpanMapperOriginSystem},
{Key: "llm.model_name", Context: spantypes.FieldContextSpanAttribute, Operation: spantypes.SpanMapperOperationMove, Priority: 1, Enabled: true, Origin: spantypes.SpanMapperOriginUser},
}}, &off, testUser))
// Add a user source to the mapper v2 stops shipping, so it must survive.
messages := findMapper(t, mappers, "gen_ai.input.messages")
require.NoError(t, v1.UpdateMapper(ctx, orgID, group.ID, messages.ID, spantypes.FieldContext{}, &spantypes.SpanMapperConfig{Sources: []spantypes.SpanMapperSource{
{Key: "input.value", Context: spantypes.FieldContextSpanAttribute, Operation: spantypes.SpanMapperOperationCopy, Priority: 1, Enabled: true},
}}, nil, testUser))
// A user mapper in the shipped group.
require.NoError(t, v1.CreateMapper(ctx, orgID, group.ID, spantypes.NewSpanMapper(group.ID, testUser, &spantypes.PostableSpanMapper{
Name: "gen_ai.custom", FieldContext: spantypes.FieldContextSpanAttribute, Enabled: true,
Config: spantypes.SpanMapperConfig{Sources: []spantypes.SpanMapperSource{{Key: "custom", Context: spantypes.FieldContextSpanAttribute, Operation: spantypes.SpanMapperOperationCopy, Priority: 1, Enabled: true}}},
})))
v2 := newTestModule(t, sqlStore, newTestDefinition(t, 2, llmV2))
require.NoError(t, v2.ReconcileSystemGroups(ctx, orgID))
upgraded, err := v2.GetGroup(ctx, orgID, group.ID)
require.NoError(t, err)
assert.Equal(t, 2, upgraded.Version)
assert.False(t, upgraded.Enabled)
assert.Equal(t, spantypes.ProvisionerIdentity, upgraded.UpdatedBy)
assert.Equal(t, []spantypes.SpanMapperGroupConditionKey{
{Value: "model", Enabled: false, Origin: spantypes.SpanMapperOriginSystem},
{Value: "llm.", Enabled: true, Origin: spantypes.SpanMapperOriginSystem},
{Value: "gen_ai.request.model", Enabled: true, Origin: spantypes.SpanMapperOriginUser},
}, upgraded.Condition.Attributes)
mappers, err = v2.ListMappers(ctx, orgID, group.ID)
require.NoError(t, err)
require.Len(t, mappers, 4)
model = findMapper(t, mappers, "gen_ai.request.model")
assert.False(t, model.Enabled)
assert.Equal(t, spantypes.SpanMapperOriginSystem, model.Origin)
assert.False(t, findSource(t, model.Config.Sources, "llm.model_name", spantypes.SpanMapperOriginSystem).Enabled)
assert.True(t, findSource(t, model.Config.Sources, "langfuse.observation.model.name", spantypes.SpanMapperOriginSystem).Enabled)
assert.Equal(t, spantypes.SpanMapperOperationMove, findSource(t, model.Config.Sources, "llm.model_name", spantypes.SpanMapperOriginUser).Operation)
assert.Len(t, model.Config.Sources, 3)
messages = findMapper(t, mappers, "gen_ai.input.messages")
assert.Equal(t, spantypes.SpanMapperOriginUser, messages.Origin)
require.Len(t, messages.Config.Sources, 1)
assert.Equal(t, "input.value", messages.Config.Sources[0].Key)
assert.Equal(t, spantypes.SpanMapperOriginSystem, findMapper(t, mappers, "gen_ai.provider.name").Origin)
assert.Equal(t, spantypes.SpanMapperOriginUser, findMapper(t, mappers, "gen_ai.custom").Origin)
// Shipping v1 again drops provider.name outright (no user sources) and
// re-adopts the surviving user mapper input.messages as a shipped one.
v3 := newTestModule(t, sqlStore, newTestDefinition(t, 3, llmV1))
require.NoError(t, v3.ReconcileSystemGroups(ctx, orgID))
mappers, err = v3.ListMappers(ctx, orgID, group.ID)
require.NoError(t, err)
require.Len(t, mappers, 3)
for _, m := range mappers {
assert.NotEqual(t, "gen_ai.provider.name", m.Name)
}
messages = findMapper(t, mappers, "gen_ai.input.messages")
assert.Equal(t, spantypes.SpanMapperOriginSystem, messages.Origin)
assert.Len(t, messages.Config.Sources, 2)
}
func TestReconcileDoesNotDowngrade(t *testing.T) {
ctx := context.Background()
orgID := valuer.GenerateUUID()
sqlStore := newTestSQLStore(t)
require.NoError(t, newTestModule(t, sqlStore, newTestDefinition(t, 2, llmV2)).ReconcileSystemGroups(ctx, orgID))
older := newTestModule(t, sqlStore, newTestDefinition(t, 1, llmV1))
require.NoError(t, older.ReconcileSystemGroups(ctx, orgID))
group, err := older.store.GetGroupByName(ctx, orgID, "llm")
require.NoError(t, err)
assert.Equal(t, 2, group.Version)
mappers, err := older.ListMappers(ctx, orgID, group.ID)
require.NoError(t, err)
findMapper(t, mappers, "gen_ai.provider.name")
}

View File

@@ -1,187 +0,0 @@
package implspanmapper
import (
"context"
"log/slog"
"slices"
"time"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/query-service/agentConf"
"github.com/SigNoz/signoz/pkg/types/spantypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
func (module *module) ReconcileSystemGroups(ctx context.Context, orgID valuer.UUID) error {
for _, definition := range module.registry.List() {
if err := module.reconcileSystemGroup(ctx, orgID, definition); err != nil {
return err
}
}
agentConf.NotifyConfigUpdate(ctx)
return nil
}
// reconcileSystemGroup brings one org's copy of a definition to the shipped
// version in a single transaction. A concurrent provisioner (another replica,
// or the org-creation hook racing the startup sweep) loses on the group's
// unique (org_id, name) index and is treated as a no-op.
func (module *module) reconcileSystemGroup(ctx context.Context, orgID valuer.UUID, definition spantypes.SpanMapperGroupDefinition) error {
err := module.store.RunInTx(ctx, func(ctx context.Context) error {
group, err := module.store.GetGroupByName(ctx, orgID, definition.Name())
if err != nil && errors.Ast(err, errors.TypeNotFound) {
group = newSystemGroup(orgID, definition)
err = module.store.CreateGroup(ctx, group)
}
if err != nil {
return err
}
if group.Origin != spantypes.SpanMapperOriginSystem {
module.settings.Logger().WarnContext(ctx, "skipping default span mapper group: a user group holds its name", slog.String("name", definition.Name()), slog.String("org_id", orgID.StringValue()))
return nil
}
if group.Version >= definition.Version {
return nil
}
return module.applyDefinition(ctx, orgID, group, definition)
})
if err != nil && errors.Ast(err, errors.TypeAlreadyExists) {
module.settings.Logger().DebugContext(ctx, "default span mapper group provisioned concurrently", slog.String("name", definition.Name()), slog.String("org_id", orgID.StringValue()))
return nil
}
return err
}
// applyDefinition replaces every shipped item with the definition, carrying each
// enabled flag over by identity, and leaves user items untouched. A mapper that
// is no longer shipped is deleted unless the user added sources to it, in which
// case it survives as a user mapper.
func (module *module) applyDefinition(ctx context.Context, orgID valuer.UUID, group *spantypes.SpanMapperGroup, definition spantypes.SpanMapperGroupDefinition) error {
mappers, err := module.store.ListMappers(ctx, orgID, group.ID)
if err != nil {
return err
}
byName := make(map[string]*spantypes.SpanMapper, len(mappers))
for _, m := range mappers {
byName[m.Name] = m
}
now := time.Now()
for i := range definition.Definition.Mappers {
pm := &definition.Definition.Mappers[i]
mapper, exists := byName[pm.Name]
delete(byName, pm.Name)
if !exists {
if err := module.store.CreateMapper(ctx, newSystemMapper(group.ID, pm)); err != nil {
return err
}
continue
}
mapper.Config.Sources = mergeShippedSources(mapper.Config.Sources, pm.Config.Sources)
mapper.FieldContext = pm.FieldContext
mapper.Origin = spantypes.SpanMapperOriginSystem
mapper.UpdatedAt = now
mapper.UpdatedBy = spantypes.ProvisionerIdentity
if err := module.store.UpdateMapper(ctx, mapper); err != nil {
return err
}
}
// Whatever is left in byName is not shipped any more.
for _, mapper := range byName {
if mapper.Origin != spantypes.SpanMapperOriginSystem {
continue
}
mapper.Config.Sources = mergeShippedSources(mapper.Config.Sources, nil)
if len(mapper.Config.Sources) == 0 {
if err := module.store.DeleteMapper(ctx, orgID, group.ID, mapper.ID); err != nil {
return err
}
continue
}
mapper.Origin = spantypes.SpanMapperOriginUser
mapper.UpdatedAt = now
mapper.UpdatedBy = spantypes.ProvisionerIdentity
if err := module.store.UpdateMapper(ctx, mapper); err != nil {
return err
}
}
shipped := definition.Definition.Condition
group.Condition = spantypes.SpanMapperGroupCondition{
Attributes: mergeShippedConditionKeys(group.Condition.Attributes, shipped.Attributes),
Resource: mergeShippedConditionKeys(group.Condition.Resource, shipped.Resource),
}
group.Version = definition.Version
group.UpdatedAt = now
group.UpdatedBy = spantypes.ProvisionerIdentity
if err := module.store.UpdateGroup(ctx, group); err != nil {
return err
}
module.settings.Logger().InfoContext(ctx, "applied default span mapper group", slog.String("name", definition.Name()), slog.Int("version", definition.Version), slog.String("org_id", orgID.StringValue()))
return nil
}
// newSystemGroup is the empty shell applyDefinition fills: version 0 so the
// definition is applied right after the row exists.
func newSystemGroup(orgID valuer.UUID, definition spantypes.SpanMapperGroupDefinition) *spantypes.SpanMapperGroup {
group := spantypes.NewSpanMapperGroup(orgID, spantypes.ProvisionerIdentity, &definition.Definition.PostableSpanMapperGroup)
group.Condition = definition.Definition.Condition
group.Enabled = true
group.Origin = spantypes.SpanMapperOriginSystem
return group
}
func newSystemMapper(groupID valuer.UUID, pm *spantypes.PostableSpanMapper) *spantypes.SpanMapper {
mapper := spantypes.NewSpanMapper(groupID, spantypes.ProvisionerIdentity, pm)
mapper.Config = pm.Config
mapper.Enabled = true
mapper.Origin = spantypes.SpanMapperOriginSystem
return mapper
}
// mergeShippedConditionKeys returns the shipped keys, each keeping the enabled
// flag of the stored system key with the same value, followed by the stored
// user keys.
func mergeShippedConditionKeys(stored, shipped []spantypes.SpanMapperGroupConditionKey) []spantypes.SpanMapperGroupConditionKey {
out := make([]spantypes.SpanMapperGroupConditionKey, 0, len(stored)+len(shipped))
for _, k := range shipped {
idx := slices.IndexFunc(stored, func(s spantypes.SpanMapperGroupConditionKey) bool {
return s.Origin == spantypes.SpanMapperOriginSystem && s.Value == k.Value
})
if idx != -1 {
k.Enabled = stored[idx].Enabled
}
out = append(out, k)
}
for _, k := range stored {
if k.Origin != spantypes.SpanMapperOriginSystem {
out = append(out, k)
}
}
return out
}
// mergeShippedSources returns the shipped sources, each keeping the enabled
// flag of the stored system source with the same key and context, followed by
// the stored user sources.
func mergeShippedSources(stored, shipped []spantypes.SpanMapperSource) []spantypes.SpanMapperSource {
out := make([]spantypes.SpanMapperSource, 0, len(stored)+len(shipped))
for _, s := range shipped {
idx := slices.IndexFunc(stored, func(o spantypes.SpanMapperSource) bool {
return o.Origin == spantypes.SpanMapperOriginSystem && o.Key == s.Key && o.Context == s.Context
})
if idx != -1 {
s.Enabled = stored[idx].Enabled
}
out = append(out, s)
}
for _, s := range stored {
if s.Origin != spantypes.SpanMapperOriginSystem {
out = append(out, s)
}
}
return out
}

View File

@@ -1,81 +0,0 @@
package implspanmapper
import (
"context"
"log/slog"
"time"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/modules/organization"
"github.com/SigNoz/signoz/pkg/modules/spanmapper"
)
const reconcileRetryInterval = 30 * time.Second
type service struct {
settings factory.ScopedProviderSettings
module spanmapper.Module
orgGetter organization.Getter
stopC chan struct{}
healthyC chan struct{}
}
// NewService reconciles every org's default mapping groups once at startup.
// Orgs created later are reconciled by the organization setter instead.
func NewService(providerSettings factory.ProviderSettings, module spanmapper.Module, orgGetter organization.Getter) factory.Service {
return &service{
settings: factory.NewScopedProviderSettings(providerSettings, "github.com/SigNoz/signoz/pkg/modules/spanmapper/implspanmapper"),
module: module,
orgGetter: orgGetter,
stopC: make(chan struct{}),
healthyC: make(chan struct{}),
}
}
func (service *service) Start(ctx context.Context) error {
ticker := time.NewTicker(reconcileRetryInterval)
defer ticker.Stop()
for {
err := service.reconcile(ctx)
if err == nil {
close(service.healthyC)
<-service.stopC
return nil
}
service.settings.Logger().WarnContext(ctx, "default span mapper group reconciliation failed, retrying", errors.Attr(err))
select {
case <-service.stopC:
return nil
case <-ticker.C:
}
}
}
func (service *service) Healthy() <-chan struct{} {
return service.healthyC
}
func (service *service) Stop(_ context.Context) error {
close(service.stopC)
return nil
}
func (service *service) reconcile(ctx context.Context) error {
orgs, err := service.orgGetter.ListByOwnedKeyRange(ctx)
if err != nil {
return err
}
for _, org := range orgs {
if err := service.module.ReconcileSystemGroups(ctx, org.ID); err != nil {
return errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "couldn't reconcile default span mapper groups for org %s", org.ID.StringValue())
}
}
service.settings.Logger().InfoContext(ctx, "default span mapper group reconciliation completed", slog.Int("orgs", len(orgs)))
return nil
}

View File

@@ -17,10 +17,6 @@ func NewStore(sqlstore sqlstore.SQLStore) spantypes.SpanMapperStore {
return &store{sqlstore: sqlstore}
}
func (s *store) RunInTx(ctx context.Context, cb func(ctx context.Context) error) error {
return s.sqlstore.RunInTxCtx(ctx, nil, cb)
}
func (s *store) CreateGroup(ctx context.Context, group *spantypes.SpanMapperGroup) error {
storable := group.ToStorable()
_, err := s.sqlstore.
@@ -38,7 +34,7 @@ func (s *store) GetGroup(ctx context.Context, orgID, id valuer.UUID) (*spantypes
storable := new(spantypes.StorableSpanMapperGroup)
err := s.sqlstore.
BunDBCtx(ctx).
BunDB().
NewSelect().
Model(storable).
Where("org_id = ?", orgID).
@@ -50,27 +46,11 @@ func (s *store) GetGroup(ctx context.Context, orgID, id valuer.UUID) (*spantypes
return storable.ToSpanMapperGroup(), nil
}
func (s *store) GetGroupByName(ctx context.Context, orgID valuer.UUID, name string) (*spantypes.SpanMapperGroup, error) {
storable := new(spantypes.StorableSpanMapperGroup)
err := s.sqlstore.
BunDBCtx(ctx).
NewSelect().
Model(storable).
Where("org_id = ?", orgID).
Where("name = ?", name).
Scan(ctx)
if err != nil {
return nil, s.sqlstore.WrapNotFoundErrf(err, spantypes.ErrCodeMappingGroupNotFound, "span mapper group %q not found", name)
}
return storable.ToSpanMapperGroup(), nil
}
func (s *store) ListGroups(ctx context.Context, orgID valuer.UUID, q *spantypes.ListSpanMapperGroupsQuery) ([]*spantypes.SpanMapperGroup, error) {
storables := make([]*spantypes.StorableSpanMapperGroup, 0)
sel := s.sqlstore.
BunDBCtx(ctx).
BunDB().
NewSelect().
Model(&storables).
Where("org_id = ?", orgID)
@@ -111,35 +91,38 @@ func (s *store) UpdateGroup(ctx context.Context, group *spantypes.SpanMapperGrou
}
func (s *store) DeleteGroup(ctx context.Context, orgID, id valuer.UUID) error {
return s.RunInTx(ctx, func(ctx context.Context) error {
db := s.sqlstore.BunDBCtx(ctx)
tx, err := s.sqlstore.BunDBCtx(ctx).BeginTx(ctx, nil)
if err != nil {
return err
}
defer func() { _ = tx.Rollback() }()
// Cascade: remove mappers belonging to this group first.
if _, err := db.NewDelete().
Model((*spantypes.StorableSpanMapper)(nil)).
Where("group_id = ?", id).
Exec(ctx); err != nil {
return err
}
// Cascade: remove mappers belonging to this group first.
if _, err := tx.NewDelete().
Model((*spantypes.StorableSpanMapper)(nil)).
Where("group_id = ?", id).
Exec(ctx); err != nil {
return err
}
res, err := db.NewDelete().
Model((*spantypes.StorableSpanMapperGroup)(nil)).
Where("org_id = ?", orgID).
Where("id = ?", id).
Exec(ctx)
if err != nil {
return err
}
res, err := tx.NewDelete().
Model((*spantypes.StorableSpanMapperGroup)(nil)).
Where("org_id = ?", orgID).
Where("id = ?", id).
Exec(ctx)
if err != nil {
return err
}
rowsAffected, err := res.RowsAffected()
if err != nil {
return err
}
if rowsAffected == 0 {
return errors.Newf(errors.TypeNotFound, spantypes.ErrCodeMappingGroupNotFound, "span mapper group %s not found", id)
}
return nil
})
rowsAffected, err := res.RowsAffected()
if err != nil {
return err
}
if rowsAffected == 0 {
return errors.Newf(errors.TypeNotFound, spantypes.ErrCodeMappingGroupNotFound, "span mapper group %s not found", id)
}
return tx.Commit()
}
func (s *store) CreateMapper(ctx context.Context, mapper *spantypes.SpanMapper) error {
@@ -163,7 +146,7 @@ func (s *store) GetMapper(ctx context.Context, orgID, groupID, id valuer.UUID) (
storable := new(spantypes.StorableSpanMapper)
err := s.sqlstore.
BunDBCtx(ctx).
BunDB().
NewSelect().
Model(storable).
Where("group_id = ?", groupID).
@@ -183,7 +166,7 @@ func (s *store) ListMappers(ctx context.Context, orgID, groupID valuer.UUID) ([]
storables := make([]*spantypes.StorableSpanMapper, 0)
if err := s.sqlstore.
BunDBCtx(ctx).
BunDB().
NewSelect().
Model(&storables).
Where("group_id = ?", groupID).

View File

@@ -28,10 +28,6 @@ type Module interface {
UpdateMapper(ctx context.Context, orgID, groupID, id valuer.UUID, fieldContext spantypes.FieldContext, config *spantypes.SpanMapperConfig, enabled *bool, updatedBy string) error
DeleteMapper(ctx context.Context, orgID, groupID, id valuer.UUID) error
TestMappers(ctx context.Context, orgID valuer.UUID, spans []spantypes.SpanMapperTestSpan, groups []*spantypes.SpanMapperGroupWithMappers) ([]spantypes.SpanMapperTestSpan, []string, error)
// ReconcileSystemGroups provisions or upgrades the shipped mapping groups
// for one org. It runs at startup for every org and again on org creation.
ReconcileSystemGroups(ctx context.Context, orgID valuer.UUID) error
}
// Handler defines the HTTP handler interface for mapping group and mapper endpoints.

View File

@@ -6,6 +6,7 @@ import (
"net/http"
nethttppprof "net/http/pprof"
runtimepprof "runtime/pprof"
"time"
"github.com/SigNoz/signoz/pkg/factory"
httpserver "github.com/SigNoz/signoz/pkg/http/server"
@@ -23,7 +24,7 @@ func NewFactory() factory.ProviderFactory[pprof.PProf, pprof.Config] {
func New(_ context.Context, settings factory.ProviderSettings, config pprof.Config) (pprof.PProf, error) {
server, err := httpserver.New(
settings.Logger.With(slog.String("pkg", "github.com/SigNoz/signoz/pkg/pprof/httppprof")),
httpserver.Config{Address: config.Address},
httpserver.Config{Address: config.Address, ReadTimeout: 10 * time.Second, WriteTimeout: 10 * time.Second},
newHandler(),
)
if err != nil {

View File

@@ -4070,20 +4070,20 @@ func (aH *APIHandler) RegisterTraceFunnelsRoutes(router *mux.Router, am *middlew
Methods(http.MethodPut)
// Analytics endpoints
traceFunnelsRouter.HandleFunc("/{funnel_id}/analytics/validate", aH.handleValidateTraces).Methods("POST")
traceFunnelsRouter.HandleFunc("/{funnel_id}/analytics/overview", aH.handleFunnelAnalytics).Methods("POST")
traceFunnelsRouter.HandleFunc("/{funnel_id}/analytics/steps", aH.handleStepAnalytics).Methods("POST")
traceFunnelsRouter.HandleFunc("/{funnel_id}/analytics/steps/overview", aH.handleFunnelStepAnalytics).Methods("POST")
traceFunnelsRouter.HandleFunc("/{funnel_id}/analytics/slow-traces", aH.handleFunnelSlowTraces).Methods("POST")
traceFunnelsRouter.HandleFunc("/{funnel_id}/analytics/error-traces", aH.handleFunnelErrorTraces).Methods("POST")
traceFunnelsRouter.HandleFunc("/{funnel_id}/analytics/validate", am.ViewAccess(aH.handleValidateTraces)).Methods("POST")
traceFunnelsRouter.HandleFunc("/{funnel_id}/analytics/overview", am.ViewAccess(aH.handleFunnelAnalytics)).Methods("POST")
traceFunnelsRouter.HandleFunc("/{funnel_id}/analytics/steps", am.ViewAccess(aH.handleStepAnalytics)).Methods("POST")
traceFunnelsRouter.HandleFunc("/{funnel_id}/analytics/steps/overview", am.ViewAccess(aH.handleFunnelStepAnalytics)).Methods("POST")
traceFunnelsRouter.HandleFunc("/{funnel_id}/analytics/slow-traces", am.ViewAccess(aH.handleFunnelSlowTraces)).Methods("POST")
traceFunnelsRouter.HandleFunc("/{funnel_id}/analytics/error-traces", am.ViewAccess(aH.handleFunnelErrorTraces)).Methods("POST")
// Analytics endpoints
traceFunnelsRouter.HandleFunc("/analytics/validate", aH.handleValidateTracesWithPayload).Methods("POST")
traceFunnelsRouter.HandleFunc("/analytics/overview", aH.handleFunnelAnalyticsWithPayload).Methods("POST")
traceFunnelsRouter.HandleFunc("/analytics/steps", aH.handleStepAnalyticsWithPayload).Methods("POST")
traceFunnelsRouter.HandleFunc("/analytics/steps/overview", aH.handleFunnelStepAnalyticsWithPayload).Methods("POST")
traceFunnelsRouter.HandleFunc("/analytics/slow-traces", aH.handleFunnelSlowTracesWithPayload).Methods("POST")
traceFunnelsRouter.HandleFunc("/analytics/error-traces", aH.handleFunnelErrorTracesWithPayload).Methods("POST")
traceFunnelsRouter.HandleFunc("/analytics/validate", am.ViewAccess(aH.handleValidateTracesWithPayload)).Methods("POST")
traceFunnelsRouter.HandleFunc("/analytics/overview", am.ViewAccess(aH.handleFunnelAnalyticsWithPayload)).Methods("POST")
traceFunnelsRouter.HandleFunc("/analytics/steps", am.ViewAccess(aH.handleStepAnalyticsWithPayload)).Methods("POST")
traceFunnelsRouter.HandleFunc("/analytics/steps/overview", am.ViewAccess(aH.handleFunnelStepAnalyticsWithPayload)).Methods("POST")
traceFunnelsRouter.HandleFunc("/analytics/slow-traces", am.ViewAccess(aH.handleFunnelSlowTracesWithPayload)).Methods("POST")
traceFunnelsRouter.HandleFunc("/analytics/error-traces", am.ViewAccess(aH.handleFunnelErrorTracesWithPayload)).Methods("POST")
}
func (aH *APIHandler) handleValidateTraces(w http.ResponseWriter, r *http.Request) {

View File

@@ -2,19 +2,9 @@ package app
import (
"context"
"fmt"
"net"
"net/http"
"slices"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/queryparser"
"github.com/gorilla/handlers"
"github.com/rs/cors"
"github.com/soheilhy/cmux"
"github.com/SigNoz/signoz/pkg/http/middleware"
"github.com/SigNoz/signoz/pkg/query-service/agentConf"
"github.com/SigNoz/signoz/pkg/query-service/app/clickhouseReader"
@@ -23,31 +13,15 @@ import (
"github.com/SigNoz/signoz/pkg/query-service/app/opamp"
opAmpModel "github.com/SigNoz/signoz/pkg/query-service/app/opamp/model"
"github.com/SigNoz/signoz/pkg/signoz"
"github.com/SigNoz/signoz/pkg/web"
"log/slog"
"go.opentelemetry.io/contrib/instrumentation/github.com/gorilla/mux/otelmux"
"go.opentelemetry.io/otel/propagation"
"github.com/SigNoz/signoz/pkg/query-service/constants"
"github.com/SigNoz/signoz/pkg/query-service/healthcheck"
"github.com/SigNoz/signoz/pkg/query-service/utils"
)
// Server runs HTTP, Mux and a grpc server
// Server runs auxiliary servers (opamp) alongside the signoz apiserver
type Server struct {
config signoz.Config
signoz *signoz.SigNoz
// public http router
httpConn net.Listener
httpServer *http.Server
httpHostPort string
opampServer *opamp.Server
unavailableChannel chan healthcheck.Status
}
// NewServer creates and initializes Server
@@ -90,20 +64,20 @@ func NewServer(config signoz.Config, signoz *signoz.SigNoz) (*Server, error) {
return nil, err
}
s := &Server{
config: config,
signoz: signoz,
httpHostPort: constants.HTTPHostPort,
unavailableChannel: make(chan healthcheck.Status),
}
// Register the legacy query-service routes on the apiserver router. The
// apiserver owns the HTTP server and applies the middleware chain at serve
// time, so these routes get the same treatment as the apiserver routes.
r := signoz.APIServer.Router()
am := middleware.NewAuthZ(signoz.Instrumentation.Logger(), signoz.Modules.OrgGetter, signoz.Authz)
httpServer, err := s.createPublicServer(apiHandler, signoz.Web)
if err != nil {
return nil, err
}
s.httpServer = httpServer
apiHandler.RegisterRoutes(r, am)
apiHandler.RegisterLogsRoutes(r, am)
apiHandler.RegisterIntegrationRoutes(r, am)
apiHandler.RegisterQueryRangeV3Routes(r, am)
apiHandler.RegisterQueryRangeV4Routes(r, am)
apiHandler.RegisterMessagingQueuesRoutes(r, am)
apiHandler.RegisterThirdPartyApiRoutes(r, am)
apiHandler.RegisterTraceFunnelsRoutes(r, am)
opAmpModel.Init(signoz.SQLStore, signoz.Instrumentation.Logger(), signoz.Modules.OrgGetter)
@@ -121,6 +95,8 @@ func NewServer(config signoz.Config, signoz *signoz.SigNoz) (*Server, error) {
return nil, err
}
s := &Server{}
s.opampServer = opamp.InitializeServer(
&opAmpModel.AllAgents,
agentConfMgr,
@@ -130,146 +106,18 @@ func NewServer(config signoz.Config, signoz *signoz.SigNoz) (*Server, error) {
return s, nil
}
// HealthCheckStatus returns health check status channel a client can subscribe to
func (s Server) HealthCheckStatus() chan healthcheck.Status {
return s.unavailableChannel
}
func (s *Server) createPublicServer(api *APIHandler, web web.Web) (*http.Server, error) {
r := NewRouter()
r.Use(middleware.NewRecovery(s.signoz.Instrumentation.Logger()).Wrap)
r.Use(otelmux.Middleware(
"apiserver",
otelmux.WithMeterProvider(s.signoz.Instrumentation.MeterProvider()),
otelmux.WithTracerProvider(s.signoz.Instrumentation.TracerProvider()),
otelmux.WithPropagators(propagation.NewCompositeTextMapPropagator(propagation.Baggage{}, propagation.TraceContext{})),
otelmux.WithFilter(func(r *http.Request) bool {
return !slices.Contains([]string{"/api/v1/health"}, r.URL.Path)
}),
))
r.Use(middleware.NewIdentN(s.signoz.IdentNResolver, s.signoz.Sharder, s.signoz.Instrumentation.Logger()).Wrap)
r.Use(middleware.NewTimeout(s.signoz.Instrumentation.Logger(),
s.config.APIServer.Timeout.ExcludedRoutes,
s.config.APIServer.Timeout.Default,
s.config.APIServer.Timeout.Max,
).Wrap)
r.Use(middleware.NewResource(s.signoz.Instrumentation.Logger()).Wrap)
r.Use(middleware.NewAudit(s.signoz.Instrumentation.Logger(), s.config.APIServer.Logging.ExcludedRoutes, s.signoz.Auditor).Wrap)
r.Use(middleware.NewComment().Wrap)
am := middleware.NewAuthZ(s.signoz.Instrumentation.Logger(), s.signoz.Modules.OrgGetter, s.signoz.Authz)
api.RegisterRoutes(r, am)
api.RegisterLogsRoutes(r, am)
api.RegisterIntegrationRoutes(r, am)
api.RegisterQueryRangeV3Routes(r, am)
api.RegisterQueryRangeV4Routes(r, am)
api.RegisterMessagingQueuesRoutes(r, am)
api.RegisterThirdPartyApiRoutes(r, am)
api.RegisterTraceFunnelsRoutes(r, am)
err := s.signoz.APIServer.AddToRouter(r)
if err != nil {
return nil, err
}
c := cors.New(cors.Options{
AllowedOrigins: []string{"*"},
AllowedMethods: []string{"GET", "DELETE", "POST", "PUT", "PATCH", "OPTIONS"},
AllowedHeaders: []string{"Accept", "Authorization", "Content-Type", "cache-control"},
})
handler := c.Handler(r)
handler = handlers.CompressHandler(handler)
err = web.AddToRouter(r)
if err != nil {
return nil, err
}
routePrefix := s.config.Global.ExternalPath()
if routePrefix != "" {
prefixed := http.StripPrefix(routePrefix, handler)
handler = http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
switch req.URL.Path {
case "/api/v1/health", "/api/v2/healthz", "/api/v2/readyz", "/api/v2/livez":
r.ServeHTTP(w, req)
return
}
prefixed.ServeHTTP(w, req)
})
}
return &http.Server{
Handler: handler,
}, nil
}
// initListeners initialises listeners of the server
func (s *Server) initListeners() error {
// listen on public port
var err error
publicHostPort := s.httpHostPort
if publicHostPort == "" {
return fmt.Errorf("constants.HTTPHostPort is required")
}
s.httpConn, err = net.Listen("tcp", publicHostPort)
if err != nil {
return err
}
slog.Info(fmt.Sprintf("Query server started listening on %s...", s.httpHostPort))
return nil
}
// Start listening on http and private http port concurrently
// Start starts the opamp websocket server. The HTTP API server is started by
// the signoz registry.
func (s *Server) Start(ctx context.Context) error {
err := s.initListeners()
if err != nil {
slog.Info("Starting OpAmp Websocket server", "addr", constants.OpAmpWsEndpoint)
if err := s.opampServer.Start(constants.OpAmpWsEndpoint); err != nil {
return err
}
var httpPort int
if port, err := utils.GetPort(s.httpConn.Addr()); err == nil {
httpPort = port
}
go func() {
slog.Info("Starting HTTP server", "port", httpPort, "addr", s.httpHostPort)
switch err := s.httpServer.Serve(s.httpConn); err {
case nil, http.ErrServerClosed, cmux.ErrListenerClosed:
// normal exit, nothing to do
default:
slog.Error("Could not start HTTP server", errors.Attr(err))
}
s.unavailableChannel <- healthcheck.Unavailable
}()
go func() {
slog.Info("Starting OpAmp Websocket server", "addr", constants.OpAmpWsEndpoint)
err := s.opampServer.Start(constants.OpAmpWsEndpoint)
if err != nil {
slog.Error("opamp ws server failed to start", errors.Attr(err))
s.unavailableChannel <- healthcheck.Unavailable
}
}()
return nil
}
func (s *Server) Stop(ctx context.Context) error {
if s.httpServer != nil {
if err := s.httpServer.Shutdown(context.Background()); err != nil {
return err
}
}
s.opampServer.Stop()
return nil

View File

@@ -10,11 +10,7 @@ import (
"github.com/SigNoz/signoz/pkg/valuer"
)
const (
HTTPHostPort = "0.0.0.0:8080" // Address to serve http (query service)
PrivateHostPort = "0.0.0.0:8085" // Address to server internal services like alert manager
OpAmpWsEndpoint = "0.0.0.0:4320" // address for opamp websocket
)
const OpAmpWsEndpoint = "0.0.0.0:4320" // address for opamp websocket
const MaxAllowedPointsInTimeSeries = 300

View File

@@ -1,12 +0,0 @@
package healthcheck
const (
// Unavailable indicates the service is not able to handle requests
Unavailable Status = iota
// Ready indicates the service is ready to handle requests
Ready
// Broken indicates that the healthcheck itself is broken, not serving HTTP
Broken
)
type Status int

View File

@@ -18,7 +18,6 @@ import (
"github.com/SigNoz/signoz/pkg/modules/dashboard/impldashboard"
"github.com/SigNoz/signoz/pkg/modules/organization/implorganization"
"github.com/SigNoz/signoz/pkg/modules/retention/implretention"
"github.com/SigNoz/signoz/pkg/modules/spanmapper/implspanmapper"
"github.com/SigNoz/signoz/pkg/modules/tag/impltag"
"github.com/SigNoz/signoz/pkg/modules/user/impluser"
"github.com/SigNoz/signoz/pkg/querier"
@@ -62,10 +61,7 @@ func TestNewHandlers(t *testing.T) {
userGetter := impluser.NewGetter(impluser.NewStore(sqlstore, providerSettings), userRoleStore, flagger)
retentionGetter := implretention.NewGetter(implretention.NewStore(sqlstore))
spanMapperRegistry, err := implspanmapper.NewSystemGroupRegistry()
require.NoError(t, err)
spanMapperModule := implspanmapper.NewModule(implspanmapper.NewStore(sqlstore), flagger, spanMapperRegistry, providerSettings)
modules := NewModules(sqlstore, tokenizer, emailing, providerSettings, orgGetter, alertmanager, nil, nil, nil, nil, nil, nil, nil, queryParser, Config{}, dashboardModule, userGetter, userRoleStore, nil, nil, nil, retentionGetter, flagger, tagModule, nil, spanMapperModule)
modules := NewModules(sqlstore, tokenizer, emailing, providerSettings, orgGetter, alertmanager, nil, nil, nil, nil, nil, nil, nil, queryParser, Config{}, dashboardModule, userGetter, userRoleStore, nil, nil, nil, retentionGetter, flagger, tagModule, nil)
querierHandler := querier.NewHandler(providerSettings, nil, nil)
registryHandler := factory.NewHandler(nil)

View File

@@ -45,6 +45,7 @@ import (
"github.com/SigNoz/signoz/pkg/modules/session"
"github.com/SigNoz/signoz/pkg/modules/session/implsession"
"github.com/SigNoz/signoz/pkg/modules/spanmapper"
"github.com/SigNoz/signoz/pkg/modules/spanmapper/implspanmapper"
"github.com/SigNoz/signoz/pkg/modules/spanpercentile"
"github.com/SigNoz/signoz/pkg/modules/spanpercentile/implspanpercentile"
"github.com/SigNoz/signoz/pkg/modules/tag"
@@ -123,10 +124,9 @@ func NewModules(
fl flagger.Flagger,
tagModule tag.Module,
metricReductionRule metricreductionrule.Module,
spanMapper spanmapper.Module,
) Modules {
quickfilter := implquickfilter.NewModule(implquickfilter.NewStore(sqlstore))
orgSetter := implorganization.NewSetter(implorganization.NewStore(sqlstore), alertmanager, quickfilter, dashboard, spanMapper)
orgSetter := implorganization.NewSetter(implorganization.NewStore(sqlstore), alertmanager, quickfilter, dashboard)
// Cleanup callbacks from other modules, invoked when a user is deleted.
onDeleteUser := []user.OnDeleteUser{
dashboard.DeletePreferencesForUser,
@@ -162,7 +162,7 @@ func NewModules(
RuleStateHistory: implrulestatehistory.NewModule(implrulestatehistory.NewStore(telemetryStore, telemetryMetadataStore, providerSettings.Logger), ruleStore),
CloudIntegration: cloudIntegrationModule,
TraceDetail: impltracedetail.NewModule(impltracedetail.NewTraceStore(telemetryStore), providerSettings, config.TraceDetail),
SpanMapper: spanMapper,
SpanMapper: implspanmapper.NewModule(implspanmapper.NewStore(sqlstore), fl),
LLMPricingRule: impllmpricingrule.NewModule(impllmpricingrule.NewStore(sqlstore), fl, querier),
Tag: tagModule,
}

View File

@@ -21,7 +21,6 @@ import (
"github.com/SigNoz/signoz/pkg/modules/retention/implretention"
"github.com/SigNoz/signoz/pkg/modules/serviceaccount"
"github.com/SigNoz/signoz/pkg/modules/serviceaccount/implserviceaccount"
"github.com/SigNoz/signoz/pkg/modules/spanmapper/implspanmapper"
"github.com/SigNoz/signoz/pkg/modules/tag/impltag"
"github.com/SigNoz/signoz/pkg/modules/user/impluser"
"github.com/SigNoz/signoz/pkg/queryparser"
@@ -69,11 +68,7 @@ func TestNewModules(t *testing.T) {
retentionGetter := implretention.NewGetter(implretention.NewStore(sqlstore))
spanMapperRegistry, err := implspanmapper.NewSystemGroupRegistry()
require.NoError(t, err)
spanMapperModule := implspanmapper.NewModule(implspanmapper.NewStore(sqlstore), flagger, spanMapperRegistry, providerSettings)
modules := NewModules(sqlstore, tokenizer, emailing, providerSettings, orgGetter, alertmanager, nil, nil, nil, nil, nil, nil, nil, queryParser, Config{}, dashboardModule, userGetter, userRoleStore, serviceAccount, serviceAccountGetter, implcloudintegration.NewModule(), retentionGetter, flagger, tagModule, implmetricreductionrule.NewModule(), spanMapperModule)
modules := NewModules(sqlstore, tokenizer, emailing, providerSettings, orgGetter, alertmanager, nil, nil, nil, nil, nil, nil, nil, queryParser, Config{}, dashboardModule, userGetter, userRoleStore, serviceAccount, serviceAccountGetter, implcloudintegration.NewModule(), retentionGetter, flagger, tagModule, implmetricreductionrule.NewModule())
reflectVal := reflect.ValueOf(modules)
for i := 0; i < reflectVal.NumField(); i++ {

View File

@@ -10,12 +10,14 @@ import (
"github.com/SigNoz/signoz/pkg/alertmanager"
"github.com/SigNoz/signoz/pkg/apiserver"
"github.com/SigNoz/signoz/pkg/apiserver/signozapiserver"
"github.com/SigNoz/signoz/pkg/auditor"
"github.com/SigNoz/signoz/pkg/authz"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/gateway"
"github.com/SigNoz/signoz/pkg/global"
"github.com/SigNoz/signoz/pkg/http/handler"
"github.com/SigNoz/signoz/pkg/identn"
"github.com/SigNoz/signoz/pkg/instrumentation"
"github.com/SigNoz/signoz/pkg/licensing"
"github.com/SigNoz/signoz/pkg/modules/aiobservability"
@@ -42,9 +44,11 @@ import (
"github.com/SigNoz/signoz/pkg/prometheus"
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/ruler"
"github.com/SigNoz/signoz/pkg/sharder"
"github.com/SigNoz/signoz/pkg/statsreporter"
"github.com/SigNoz/signoz/pkg/subscription"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/web"
"github.com/SigNoz/signoz/pkg/zeus"
"github.com/swaggest/jsonschema-go"
"github.com/swaggest/openapi-go"
@@ -101,6 +105,11 @@ func NewOpenAPI(ctx context.Context, instrumentation instrumentation.Instrumenta
struct{ ruler.Handler }{},
struct{ statsreporter.Handler }{},
struct{ savedview.Handler }{},
global.Config{},
struct{ identn.IdentNResolver }{},
struct{ sharder.Sharder }{},
struct{ auditor.Auditor }{},
struct{ web.Web }{},
struct{ quickfilter.Module }{},
struct{ quickfilter.Handler }{},
).New(ctx, instrumentation.ToProviderSettings(), apiserver.Config{})

View File

@@ -254,7 +254,6 @@ func NewSQLMigrationProviderFactories(
sqlmigration.NewAddIngestionTuplesFactory(sqlstore),
sqlmigration.NewAddSubscriptionTuplesFactory(sqlstore),
sqlmigration.NewNormalizeQuickFilterFieldsFactory(sqlstore),
sqlmigration.NewAddSpanMapperOriginFactory(sqlstore, sqlschema),
)
}
@@ -320,7 +319,7 @@ func NewQuerierProviderFactories(telemetryStore telemetrystore.TelemetryStore, p
)
}
func NewAPIServerProviderFactories(orgGetter organization.Getter, authz authz.AuthZ, modules Modules, handlers Handlers, globalConfig global.Config, gatewayService gateway.Gateway) factory.NamedMap[factory.ProviderFactory[apiserver.APIServer, apiserver.Config]] {
func NewAPIServerProviderFactories(orgGetter organization.Getter, authz authz.AuthZ, modules Modules, handlers Handlers, globalConfig global.Config, gatewayService gateway.Gateway, identNResolver identn.IdentNResolver, sharder sharder.Sharder, auditor auditor.Auditor, web web.Web) factory.NamedMap[factory.ProviderFactory[apiserver.APIServer, apiserver.Config]] {
return factory.MustNewNamedMap(
signozapiserver.NewFactory(
orgGetter,
@@ -362,6 +361,11 @@ func NewAPIServerProviderFactories(orgGetter organization.Getter, authz authz.Au
handlers.RulerHandler,
handlers.StatsHandler,
handlers.SavedView,
globalConfig,
identNResolver,
sharder,
auditor,
web,
modules.QuickFilter,
handlers.QuickFilter,
),

View File

@@ -102,6 +102,10 @@ func TestNewProviderFactories(t *testing.T) {
Handlers{},
global.Config{},
nil,
nil,
nil,
nil,
nil,
)
})
}

View File

@@ -36,7 +36,6 @@ import (
"github.com/SigNoz/signoz/pkg/modules/rulestatehistory"
"github.com/SigNoz/signoz/pkg/modules/serviceaccount"
"github.com/SigNoz/signoz/pkg/modules/serviceaccount/implserviceaccount"
"github.com/SigNoz/signoz/pkg/modules/spanmapper/implspanmapper"
"github.com/SigNoz/signoz/pkg/modules/tag"
"github.com/SigNoz/signoz/pkg/modules/tag/impltag"
"github.com/SigNoz/signoz/pkg/modules/user/impluser"
@@ -550,15 +549,7 @@ func New(
metricReductionRuleModule := metricReductionRuleModuleCallback(sqlstore, telemetrystore, dashboard, queryParser, licensing, flagger, telemetryMetadataStore, providerSettings, config.MetricsExplorer.TelemetryStore.Threads)
// Initialize all modules
// The default mapping group registry is parsed here so a malformed embedded
// definition fails startup instead of a request.
spanMapperRegistry, err := implspanmapper.NewSystemGroupRegistry()
if err != nil {
return nil, err
}
spanMapperModule := implspanmapper.NewModule(implspanmapper.NewStore(sqlstore), flagger, spanMapperRegistry, providerSettings)
modules := NewModules(sqlstore, tokenizer, emailing, providerSettings, orgGetter, alertmanager, analytics, querier, telemetrystore, telemetryMetadataStore, authNs, authz, cache, queryParser, config, dashboard, userGetter, userRoleStore, serviceAccount, serviceAccountGetter, cloudIntegrationModule, retentionGetter, flagger, tagModule, metricReductionRuleModule, spanMapperModule)
modules := NewModules(sqlstore, tokenizer, emailing, providerSettings, orgGetter, alertmanager, analytics, querier, telemetrystore, telemetryMetadataStore, authNs, authz, cache, queryParser, config, dashboard, userGetter, userRoleStore, serviceAccount, serviceAccountGetter, cloudIntegrationModule, retentionGetter, flagger, tagModule, metricReductionRuleModule)
// Initialize ruler from the variant-specific provider factories
rulerInstance, err := factory.NewProviderFromNamedMap(ctx, providerSettings, config.Ruler, rulerProviderFactories(cache, alertmanager, sqlstore, telemetrystore, telemetryMetadataStore, prometheus, orgGetter, modules.RuleStateHistory, querier, queryParser), "signoz")
@@ -628,7 +619,6 @@ func New(
factory.NewNamedService(factory.MustNewName("meterreporter"), meterReporter, factory.MustNewName("licensing")),
factory.NewNamedService(factory.MustNewName("ruler"), rulerInstance),
factory.NewNamedService(factory.MustNewName("systemdashboard"), impldashboard.NewService(providerSettings, dashboard, orgGetter)),
factory.NewNamedService(factory.MustNewName("spanmappergroup"), implspanmapper.NewService(providerSettings, spanMapperModule, orgGetter)),
)
if err != nil {
return nil, err
@@ -645,13 +635,20 @@ func New(
ctx,
providerSettings,
config.APIServer,
NewAPIServerProviderFactories(orgGetter, authz, modules, handlers, config.Global, gateway),
NewAPIServerProviderFactories(orgGetter, authz, modules, handlers, config.Global, gateway, identNResolver, sharder, auditor, web),
"signoz",
)
if err != nil {
return nil, err
}
// Register the API server with the registry so its lifecycle is managed
// alongside the other services and it shows up in the health endpoint.
err = registry.Add(ctx, factory.NewNamedService(factory.MustNewName("apiserver"), apiserverInstance))
if err != nil {
return nil, err
}
return &SigNoz{
Registry: registry,
Analytics: analytics,

View File

@@ -1,93 +0,0 @@
package sqlmigration
import (
"context"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/sqlschema"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/uptrace/bun"
"github.com/uptrace/bun/migrate"
)
type addSpanMapperOrigin struct {
sqlstore sqlstore.SQLStore
sqlschema sqlschema.SQLSchema
}
func NewAddSpanMapperOriginFactory(sqlstore sqlstore.SQLStore, sqlschema sqlschema.SQLSchema) factory.ProviderFactory[SQLMigration, Config] {
return factory.NewProviderFactory(
factory.MustNewName("add_span_mapper_origin"),
func(ctx context.Context, ps factory.ProviderSettings, c Config) (SQLMigration, error) {
return &addSpanMapperOrigin{sqlstore: sqlstore, sqlschema: sqlschema}, nil
},
)
}
func (migration *addSpanMapperOrigin) Register(migrations *migrate.Migrations) error {
return migrations.Register(migration.Up, migration.Down)
}
// Up adds the ownership columns that let SigNoz ship default mapping groups
// alongside user ones.
func (migration *addSpanMapperOrigin) Up(ctx context.Context, db *bun.DB) error {
// span_mapper references span_mapper_group and both have foreign keys, so
// enforcement must be off for the SQLite recreate-table fallback.
if err := migration.sqlschema.ToggleFKEnforcement(ctx, db, false); err != nil {
return err
}
tx, err := db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer func() {
_ = tx.Rollback()
}()
groupTable, groupUniqueConstraints, err := migration.sqlschema.GetTable(ctx, sqlschema.TableName("span_mapper_group"))
if err != nil {
return err
}
sqls := migration.sqlschema.Operator().AddColumn(groupTable, groupUniqueConstraints, &sqlschema.Column{
Name: sqlschema.ColumnName("origin"),
DataType: sqlschema.DataTypeText,
Nullable: false,
Default: "'user'",
}, "user")
sqls = append(sqls, migration.sqlschema.Operator().AddColumn(groupTable, groupUniqueConstraints, &sqlschema.Column{
Name: sqlschema.ColumnName("version"),
DataType: sqlschema.DataTypeBigInt,
Nullable: false,
Default: "0",
}, 0)...)
mapperTable, mapperUniqueConstraints, err := migration.sqlschema.GetTable(ctx, sqlschema.TableName("span_mapper"))
if err != nil {
return err
}
sqls = append(sqls, migration.sqlschema.Operator().AddColumn(mapperTable, mapperUniqueConstraints, &sqlschema.Column{
Name: sqlschema.ColumnName("origin"),
DataType: sqlschema.DataTypeText,
Nullable: false,
Default: "'user'",
}, "user")...)
for _, sql := range sqls {
if _, err := tx.ExecContext(ctx, string(sql)); err != nil {
return err
}
}
if err := tx.Commit(); err != nil {
return err
}
return migration.sqlschema.ToggleFKEnforcement(ctx, db, true)
}
func (migration *addSpanMapperOrigin) Down(context.Context, *bun.DB) error {
return nil
}

View File

@@ -141,6 +141,7 @@ func (d *DashboardSpec) validateQuery(qi int, q Query, panelKind PanelPluginKind
func validateQueryAllowedForPanel(plugin QueryPlugin, allowed []QueryPluginKind, panelKind PanelPluginKind, path string) error {
compositeSubQueryTypeToPluginKind := map[qb.QueryType]QueryPluginKind{
qb.QueryTypeBuilder: QueryKindBuilder,
qb.QueryTypeBuilderAI: QueryKindAIBuilder,
qb.QueryTypeFormula: QueryKindFormula,
qb.QueryTypeTraceOperator: QueryKindTraceOperator,
qb.QueryTypePromQL: QueryKindPromQL,

View File

@@ -117,6 +117,22 @@ func TestNewStatsFromStorableDashboardsCountsCompositeSubQueries(t *testing.T) {
assert.Equal(t, int64(1), stats[statKeyPanelLogsCount])
}
// An AI builder query is always a traces query, so it counts towards traces.
func TestNewStatsFromStorableDashboardsCountsAIBuilderQueries(t *testing.T) {
aiBuilder := `{
"kind": "time_series",
"spec": {"plugin": {"kind": "signoz/AIBuilderQuery", "spec": {"name": "A", "aggregations": [{"expression": "count()"}]}}}
}`
dashboard := newStatsStorableV2(t, `"p1": `+statsPanel(aiBuilder))
stats := NewStatsFromStorableDashboards([]*StorableDashboard{dashboard})
assert.Equal(t, int64(1), stats[statKeyPanelCount])
assert.Equal(t, int64(1), stats[statKeyPanelTracesCount])
assert.Equal(t, int64(0), stats[statKeyPanelMetricsCount])
assert.Equal(t, int64(0), stats[statKeyPanelLogsCount])
}
// promql and clickhouse queries carry no signal, so they land in the panel total
// and nowhere else.
func TestNewStatsFromStorableDashboardsIgnoresSignallessQueries(t *testing.T) {

View File

@@ -1638,6 +1638,8 @@ func TestPanelTypeQueryTypeCompatibility(t *testing.T) {
{"TimeSeries+PromQL", mkQuery("signoz/TimeSeriesPanel", "signoz/PromQLQuery", `{"name":"A","query":"up"}`), false},
{"Table+ClickHouse", mkQuery("signoz/TablePanel", "signoz/ClickHouseSQL", `{"name":"A","query":"SELECT 1"}`), false},
{"List+Builder", mkQuery("signoz/ListPanel", "signoz/BuilderQuery", `{"name":"A","signal":"logs"}`), false},
{"TimeSeries+AIBuilder", mkQuery("signoz/TimeSeriesPanel", "signoz/AIBuilderQuery", `{"name":"A","aggregations":[{"expression":"count()"}]}`), false},
{"List+AIBuilder", mkQuery("signoz/ListPanel", "signoz/AIBuilderQuery", `{"name":"A"}`), false},
// Top-level: rejected
{"Table+PromQL", mkQuery("signoz/TablePanel", "signoz/PromQLQuery", `{"name":"A","query":"up"}`), true},
{"List+ClickHouse", mkQuery("signoz/ListPanel", "signoz/ClickHouseSQL", `{"name":"A","query":"SELECT 1"}`), true},
@@ -1647,6 +1649,7 @@ func TestPanelTypeQueryTypeCompatibility(t *testing.T) {
// Composite sub-queries
{"Table+Composite(promql)", mkComposite("signoz/TablePanel", "promql", `{"name":"A","query":"up"}`), true},
{"Table+Composite(clickhouse)", mkComposite("signoz/TablePanel", "clickhouse_sql", `{"name":"A","query":"SELECT 1"}`), false},
{"Table+Composite(builder_ai)", mkComposite("signoz/TablePanel", "builder_ai_query", `{"name":"A","aggregations":[{"expression":"count()"}]}`), false},
}
for _, tc := range cases {

View File

@@ -89,6 +89,7 @@ type QueryPlugin struct {
func (QueryPlugin) PrepareJSONSchema(s *jsonschema.Schema) error {
return markDiscriminator(s, "kind", map[string]string{
string(QueryKindBuilder): schemaRef("DashboardtypesQueryPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesBuilderQuerySpec"),
string(QueryKindAIBuilder): schemaRef("DashboardtypesQueryPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesAIBuilderQuerySpec"),
string(QueryKindComposite): schemaRef("DashboardtypesQueryPluginVariantGithubComSigNozSignozPkgTypesQuerybuildertypesQuerybuildertypesv5CompositeQuery"),
string(QueryKindFormula): schemaRef("DashboardtypesQueryPluginVariantGithubComSigNozSignozPkgTypesQuerybuildertypesQuerybuildertypesv5QueryBuilderFormula"),
string(QueryKindPromQL): schemaRef("DashboardtypesQueryPluginVariantGithubComSigNozSignozPkgTypesQuerybuildertypesQuerybuildertypesv5PromQuery"),
@@ -118,6 +119,7 @@ func (p *QueryPlugin) UnmarshalJSON(data []byte) error {
func (QueryPlugin) JSONSchemaOneOf() []any {
return []any{
QueryPluginVariant[BuilderQuerySpec]{Kind: string(QueryKindBuilder)},
QueryPluginVariant[AIBuilderQuerySpec]{Kind: string(QueryKindAIBuilder)},
QueryPluginVariant[CompositeQuerySpec]{Kind: string(QueryKindComposite)},
QueryPluginVariant[FormulaSpec]{Kind: string(QueryKindFormula)},
QueryPluginVariant[PromQLQuerySpec]{Kind: string(QueryKindPromQL)},
@@ -138,6 +140,11 @@ func (plugin QueryPlugin) buildV5CompositeQueryFromPlugin() (qb.CompositeQuery,
return qb.CompositeQuery{}, errors.Newf(errors.TypeInvalidInput, ErrCodeDashboardInvalidWidgetQuery, "builder query is empty")
}
return wrapEnvelope(qb.QueryTypeBuilder, spec.Spec), nil
case *AIBuilderQuerySpec:
if spec == nil {
return qb.CompositeQuery{}, errors.Newf(errors.TypeInvalidInput, ErrCodeDashboardInvalidWidgetQuery, "AI builder query is empty")
}
return wrapEnvelope(qb.QueryTypeBuilderAI, qb.QueryBuilderQuery[qb.TraceAggregation](*spec)), nil
case *qb.PromQuery:
return wrapEnvelope(qb.QueryTypePromQL, *spec), nil
case *qb.ClickHouseQuery:
@@ -231,6 +238,7 @@ var (
}
queryPluginSpecs = map[QueryPluginKind]func() any{
QueryKindBuilder: func() any { return new(BuilderQuerySpec) },
QueryKindAIBuilder: func() any { return new(AIBuilderQuerySpec) },
QueryKindComposite: func() any { return new(CompositeQuerySpec) },
QueryKindFormula: func() any { return new(FormulaSpec) },
QueryKindPromQL: func() any { return new(PromQLQuerySpec) },
@@ -243,13 +251,13 @@ var (
VariableKindCustom: func() any { return new(CustomVariableSpec) },
}
allowedQueryKinds = map[PanelPluginKind][]QueryPluginKind{
PanelKindTimeSeries: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindPromQL, QueryKindClickHouseSQL},
PanelKindBarChart: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindPromQL, QueryKindClickHouseSQL},
PanelKindNumber: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindPromQL, QueryKindClickHouseSQL},
PanelKindHistogram: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindPromQL, QueryKindClickHouseSQL},
PanelKindPieChart: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindClickHouseSQL},
PanelKindTable: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindClickHouseSQL},
PanelKindList: {QueryKindBuilder},
PanelKindTimeSeries: {QueryKindBuilder, QueryKindAIBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindPromQL, QueryKindClickHouseSQL},
PanelKindBarChart: {QueryKindBuilder, QueryKindAIBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindPromQL, QueryKindClickHouseSQL},
PanelKindNumber: {QueryKindBuilder, QueryKindAIBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindPromQL, QueryKindClickHouseSQL},
PanelKindHistogram: {QueryKindBuilder, QueryKindAIBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindPromQL, QueryKindClickHouseSQL},
PanelKindPieChart: {QueryKindBuilder, QueryKindAIBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindClickHouseSQL},
PanelKindTable: {QueryKindBuilder, QueryKindAIBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindClickHouseSQL},
PanelKindList: {QueryKindBuilder, QueryKindAIBuilder},
}
)

View File

@@ -106,6 +106,13 @@ func redactQuery(spec any) any {
return spec
}
return &BuilderQuerySpec{Spec: redactLeafQuery(s.Spec)}
case *AIBuilderQuerySpec:
if s == nil {
return spec
}
redacted := redactLeafQuery(qb.QueryBuilderQuery[qb.TraceAggregation](*s)).(qb.QueryBuilderQuery[qb.TraceAggregation])
out := AIBuilderQuerySpec(redacted)
return &out
case *qb.PromQuery:
return redactQueryPtr(s)
case *qb.ClickHouseQuery:

View File

@@ -159,6 +159,11 @@ func TestDashboardV2GetPanelQuery(t *testing.T) {
plugin QueryPlugin
expectedType qb.QueryType
}{
{
description: "AI builder query",
plugin: QueryPlugin{Kind: QueryKindAIBuilder, Spec: &AIBuilderQuerySpec{Name: "A"}},
expectedType: qb.QueryTypeBuilderAI,
},
{
description: "promql",
plugin: QueryPlugin{Kind: QueryKindPromQL, Spec: &qb.PromQuery{Name: "A", Query: "up"}},

View File

@@ -133,6 +133,19 @@ func TestRedactQueryPluginWrappers(t *testing.T) {
assert.Equal(t, "A", builder.Name)
})
t.Run("AI builder plugin pointer is redacted and stays a pointer", func(t *testing.T) {
plugin := &AIBuilderQuerySpec{
Name: "A",
Filter: &qb.Filter{Expression: "body contains 'secret'"},
}
result, ok := redactQuery(plugin).(*AIBuilderQuerySpec)
require.True(t, ok)
assert.Nil(t, result.Filter)
assert.Equal(t, "A", result.Name)
})
t.Run("composite plugin redacts every sub-query envelope", func(t *testing.T) {
composite := &qb.CompositeQuery{Queries: []qb.QueryEnvelope{
{Type: qb.QueryTypeBuilder, Spec: qb.QueryBuilderQuery[qb.MetricAggregation]{Name: "A", Filter: &qb.Filter{Expression: "x = 1"}}},

View File

@@ -93,6 +93,7 @@ type QueryPluginKind string
const (
QueryKindBuilder QueryPluginKind = "signoz/BuilderQuery"
QueryKindAIBuilder QueryPluginKind = "signoz/AIBuilderQuery"
QueryKindComposite QueryPluginKind = "signoz/CompositeQuery"
QueryKindFormula QueryPluginKind = "signoz/Formula"
QueryKindPromQL QueryPluginKind = "signoz/PromQLQuery"
@@ -101,7 +102,7 @@ const (
)
func (QueryPluginKind) Enum() []any {
return []any{QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindPromQL, QueryKindClickHouseSQL, QueryKindTraceOperator}
return []any{QueryKindBuilder, QueryKindAIBuilder, QueryKindComposite, QueryKindFormula, QueryKindPromQL, QueryKindClickHouseSQL, QueryKindTraceOperator}
}
type (
@@ -159,6 +160,26 @@ func (BuilderQuerySpec) JSONSchemaOneOf() []any {
}
}
// AIBuilderQuerySpec is the spec of a signoz/AIBuilderQuery plugin: a gen_ai-scoped
// (AI observability) traces builder query, executed as qb.QueryTypeBuilderAI. The
// signal is implied by the kind and pinned to traces, mirroring the builder_ai_query
// QueryEnvelope decode.
type AIBuilderQuerySpec qb.QueryBuilderQuery[qb.TraceAggregation]
func (b *AIBuilderQuerySpec) UnmarshalJSON(data []byte) error {
var spec qb.QueryBuilderQuery[qb.TraceAggregation]
if err := json.Unmarshal(data, &spec); err != nil {
return errors.WrapInvalidInputf(err, ErrCodeDashboardInvalidInput, "invalid AI builder query spec")
}
spec.Signal = telemetrytypes.SignalTraces
*b = AIBuilderQuerySpec(spec)
return nil
}
func (AIBuilderQuerySpec) PrepareJSONSchema(s *jsonschema.Schema) error {
return (qb.QueryBuilderQuery[qb.TraceAggregation]{}).PrepareJSONSchema(s)
}
// ══════════════════════════════════════════════
// SigNoz panel plugin specs
// ══════════════════════════════════════════════

View File

@@ -0,0 +1,75 @@
package dashboardtypes
import (
"encoding/json"
"testing"
"github.com/SigNoz/signoz/pkg/errors"
qb "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// An AI builder query is a gen_ai-scoped traces builder query: the signal is
// implied by the plugin kind and pinned to traces on decode, mirroring the
// builder_ai_query QueryEnvelope.
func TestAIBuilderQueryPluginRoundTrip(t *testing.T) {
data := []byte(`{
"variables": [],
"panels": {"p1": {"kind": "Panel", "spec": {
"links": [],
"plugin": {"kind": "signoz/TimeSeriesPanel", "spec": {}},
"queries": [{"kind": "time_series", "spec": {"plugin": {"kind": "signoz/AIBuilderQuery", "spec": {
"name": "A", "aggregations": [{"expression": "count()"}]
}}}}]
}}},
"links": [],
"layouts": []
}`)
spec, err := unmarshalDashboard(data)
require.NoError(t, err)
plugin := spec.Panels["p1"].Spec.Queries[0].Spec.Plugin
assert.Equal(t, QueryKindAIBuilder, plugin.Kind)
aiSpec, ok := plugin.Spec.(*AIBuilderQuerySpec)
require.True(t, ok, "expected *AIBuilderQuerySpec, got %T", plugin.Spec)
assert.Equal(t, "A", aiSpec.Name)
assert.Equal(t, telemetrytypes.SignalTraces, aiSpec.Signal)
// Marshal emits the pinned signal and decodes back to the same plugin.
out, err := json.Marshal(plugin)
require.NoError(t, err)
assert.Contains(t, string(out), `"kind":"signoz/AIBuilderQuery"`)
assert.Contains(t, string(out), `"signal":"traces"`)
var roundTripped QueryPlugin
require.NoError(t, json.Unmarshal(out, &roundTripped))
assert.Equal(t, plugin, roundTripped)
}
// At query-range time an AI builder query wraps into a single builder_ai_query
// envelope so the querier routes it to the gen_ai statement builder.
func TestAIBuilderQueryPluginBuildsBuilderAIEnvelope(t *testing.T) {
plugin := QueryPlugin{Kind: QueryKindAIBuilder, Spec: &AIBuilderQuerySpec{Name: "A", Signal: telemetrytypes.SignalTraces}}
composite, err := plugin.buildV5CompositeQueryFromPlugin()
require.NoError(t, err)
require.Len(t, composite.Queries, 1)
assert.Equal(t, qb.QueryTypeBuilderAI, composite.Queries[0].Type)
spec, ok := composite.Queries[0].Spec.(qb.QueryBuilderQuery[qb.TraceAggregation])
require.True(t, ok, "expected traces builder query, got %T", composite.Queries[0].Spec)
assert.Equal(t, "A", spec.Name)
assert.Equal(t, telemetrytypes.SignalTraces, spec.Signal)
}
func TestAIBuilderQueryPluginNilSpec(t *testing.T) {
plugin := QueryPlugin{Kind: QueryKindAIBuilder, Spec: (*AIBuilderQuerySpec)(nil)}
_, err := plugin.buildV5CompositeQueryFromPlugin()
require.Error(t, err)
assert.True(t, errors.Ast(err, errors.TypeInvalidInput))
}

View File

@@ -1,8 +1,6 @@
package spantypes
import (
"slices"
"strings"
"time"
"github.com/SigNoz/signoz/pkg/errors"
@@ -13,7 +11,6 @@ import (
var (
ErrCodeMapperNotFound = errors.MustNewCode("span_attribute_mapper_not_found")
ErrCodeMapperAlreadyExists = errors.MustNewCode("span_attribute_mapper_already_exists")
ErrCodeMapperNotDeletable = errors.MustNewCode("span_attribute_mapper_not_deletable")
ErrCodeMappingInvalidInput = errors.MustNewCode("span_attribute_mapping_invalid_input")
)
@@ -37,25 +34,12 @@ var (
SpanMapperOperationCopy = SpanMapperOperation{valuer.NewString("copy")}
)
// SpanMapperOrigin tells shipped (system) items apart from user-created ones.
// System items are read-only apart from their enabled toggle.
type SpanMapperOrigin struct {
valuer.String
}
var (
SpanMapperOriginUser = SpanMapperOrigin{valuer.NewString("user")}
SpanMapperOriginSystem = SpanMapperOrigin{valuer.NewString("system")}
)
// MapperSource describes one candidate source for a target attribute.
type SpanMapperSource struct {
Key string `json:"key" required:"true"`
Context FieldContext `json:"context" required:"true"`
Operation SpanMapperOperation `json:"operation" required:"true"`
Priority int `json:"priority" required:"true"`
Enabled bool `json:"enabled" required:"true"`
Origin SpanMapperOrigin `json:"origin"`
}
// MapperConfig holds the mapping logic for a single target attribute.
@@ -75,7 +59,6 @@ type SpanMapper struct {
FieldContext FieldContext `json:"fieldContext" required:"true"`
Config SpanMapperConfig `json:"config" required:"true"`
Enabled bool `json:"enabled" required:"true"`
Origin SpanMapperOrigin `json:"origin" required:"true"`
}
type PostableSpanMapper struct {
@@ -107,63 +90,6 @@ func (SpanMapperOperation) Enum() []any {
return []any{SpanMapperOperationMove, SpanMapperOperationCopy}
}
func (SpanMapperOrigin) Enum() []any {
return []any{SpanMapperOriginUser, SpanMapperOriginSystem}
}
func (p *PostableSpanMapper) Validate() error {
if strings.TrimSpace(p.Name) == "" {
return errors.New(errors.TypeInvalidInput, ErrCodeMappingInvalidInput, "mapper name must not be blank")
}
if err := p.FieldContext.Validate(); err != nil {
return err
}
return p.Config.Validate()
}
func (f FieldContext) Validate() error {
if f != FieldContextSpanAttribute && f != FieldContextResource {
return errors.Newf(errors.TypeInvalidInput, ErrCodeMappingInvalidInput, "field context must be one of %q or %q, got %q", FieldContextSpanAttribute, FieldContextResource, f.StringValue())
}
return nil
}
// Validate checks every source and rejects duplicate priorities within an
// origin. Shipped and user sources are never compared with each other: a user
// re-adding a shipped key with another operation is the supported override.
func (c *SpanMapperConfig) Validate() error {
if len(c.Sources) == 0 {
return errors.New(errors.TypeInvalidInput, ErrCodeMappingInvalidInput, "config.sources must contain at least one source")
}
seen := map[SpanMapperOrigin]map[int]struct{}{}
for _, s := range c.Sources {
if strings.TrimSpace(s.Key) == "" {
return errors.New(errors.TypeInvalidInput, ErrCodeMappingInvalidInput, "source key must not be blank")
}
if err := s.Context.Validate(); err != nil {
return err
}
if s.Operation != SpanMapperOperationCopy && s.Operation != SpanMapperOperationMove {
return errors.Newf(errors.TypeInvalidInput, ErrCodeMappingInvalidInput, "source operation must be one of %q or %q, got %q", SpanMapperOperationCopy, SpanMapperOperationMove, s.Operation.StringValue())
}
if !s.Origin.IsZero() && s.Origin != SpanMapperOriginUser && s.Origin != SpanMapperOriginSystem {
return errors.Newf(errors.TypeInvalidInput, ErrCodeMappingInvalidInput, "source origin must be one of %q or %q, got %q", SpanMapperOriginUser, SpanMapperOriginSystem, s.Origin.StringValue())
}
origin := s.Origin
if origin.IsZero() {
origin = SpanMapperOriginUser
}
if seen[origin] == nil {
seen[origin] = map[int]struct{}{}
}
if _, dup := seen[origin][s.Priority]; dup {
return errors.Newf(errors.TypeInvalidInput, ErrCodeMappingInvalidInput, "source priority %d is used more than once", s.Priority)
}
seen[origin][s.Priority] = struct{}{}
}
return nil
}
func NewSpanMapper(groupID valuer.UUID, createdBy string, p *PostableSpanMapper) *SpanMapper {
now := time.Now()
return &SpanMapper{
@@ -171,9 +97,8 @@ func NewSpanMapper(groupID valuer.UUID, createdBy string, p *PostableSpanMapper)
GroupID: groupID,
Name: p.Name,
FieldContext: p.FieldContext,
Config: SpanMapperConfig{Sources: withOrigin(p.Config.Sources, SpanMapperOriginUser)},
Config: p.Config,
Enabled: p.Enabled,
Origin: SpanMapperOriginUser,
TimeAuditable: types.TimeAuditable{
CreatedAt: now,
UpdatedAt: now,
@@ -185,42 +110,16 @@ func NewSpanMapper(groupID valuer.UUID, createdBy string, p *PostableSpanMapper)
}
}
// Update applies a user edit; a zero fieldContext means it was omitted. On a
// system mapper the field context is fixed and the stored system sources are
// kept; see nextSources.
func (m *SpanMapper) Update(fieldContext FieldContext, config *SpanMapperConfig, enabled *bool, updatedBy string) error {
if !fieldContext.IsZero() {
if m.Origin == SpanMapperOriginSystem && fieldContext != m.FieldContext {
return errors.Newf(errors.TypeInvalidInput, ErrCodeMappingInvalidInput, "field context of system mapper %q cannot be changed", m.Name)
}
if err := fieldContext.Validate(); err != nil {
return err
}
m.FieldContext = fieldContext
}
func (m *SpanMapper) Update(fieldContext FieldContext, config *SpanMapperConfig, enabled *bool, updatedBy string) {
m.FieldContext = fieldContext
if config != nil {
sources, err := m.nextSources(config.Sources)
if err != nil {
return err
}
m.Config = SpanMapperConfig{Sources: sources}
if err := m.Config.Validate(); err != nil {
return err
}
m.Config = *config
}
if enabled != nil {
m.Enabled = *enabled
}
m.UpdatedAt = time.Now()
m.UpdatedBy = updatedBy
return nil
}
func (m *SpanMapper) ErrIfNotDeletable() error {
if m.Origin == SpanMapperOriginSystem {
return errors.Newf(errors.TypeInvalidInput, ErrCodeMapperNotDeletable, "system mapper %q cannot be deleted, disable it instead", m.Name)
}
return nil
}
func (m *SpanMapper) ToStorable() *StorableSpanMapper {
@@ -233,7 +132,6 @@ func (m *SpanMapper) ToStorable() *StorableSpanMapper {
FieldContext: m.FieldContext,
Config: m.Config,
Enabled: m.Enabled,
Origin: m.Origin,
}
}
@@ -247,7 +145,6 @@ func (s *StorableSpanMapper) ToSpanMapper() *SpanMapper {
FieldContext: s.FieldContext,
Config: s.Config,
Enabled: s.Enabled,
Origin: s.Origin,
}
}
@@ -262,39 +159,3 @@ func NewSpanMappersFromStorable(ss []*StorableSpanMapper) []*SpanMapper {
func NewGettableSpanMappers(m []*SpanMapper) *GettableSpanMappers {
return &GettableSpanMappers{Items: m}
}
// nextSources builds the source list from an edit: user sources are taken from
// the edit as sent, system sources stay as stored and the edit can only flip
// their enabled flag.
func (m *SpanMapper) nextSources(edit []SpanMapperSource) ([]SpanMapperSource, error) {
var systemSources, userSources []SpanMapperSource
for _, s := range m.Config.Sources {
if s.Origin == SpanMapperOriginSystem {
systemSources = append(systemSources, s)
}
}
for _, s := range edit {
if s.Origin != SpanMapperOriginSystem {
s.Origin = SpanMapperOriginUser
userSources = append(userSources, s)
continue
}
idx := slices.IndexFunc(systemSources, func(o SpanMapperSource) bool { return o.Key == s.Key && o.Context == s.Context })
if idx == -1 {
return nil, errors.Newf(errors.TypeInvalidInput, ErrCodeMappingInvalidInput, "system source %q does not exist on this mapper; only its enabled flag can change", s.Key)
}
systemSources[idx].Enabled = s.Enabled
}
return append(systemSources, userSources...), nil
}
func withOrigin(sources []SpanMapperSource, origin SpanMapperOrigin) []SpanMapperSource {
out := make([]SpanMapperSource, len(sources))
for i, s := range sources {
s.Origin = origin
out[i] = s
}
return out
}

View File

@@ -1,8 +1,6 @@
package spantypes
import (
"slices"
"strings"
"time"
"github.com/SigNoz/signoz/pkg/errors"
@@ -13,28 +11,17 @@ import (
var (
ErrCodeMappingGroupNotFound = errors.MustNewCode("span_attribute_mapping_group_not_found")
ErrCodeMappingGroupAlreadyExists = errors.MustNewCode("span_attribute_mapping_group_already_exists")
ErrCodeMappingGroupNameReserved = errors.MustNewCode("span_attribute_mapping_group_name_reserved")
ErrCodeMappingGroupNotDeletable = errors.MustNewCode("span_attribute_mapping_group_not_deletable")
)
// SpanMapperGroupConditionKey is one substring a span's attribute or resource
// keys are matched against.
type SpanMapperGroupConditionKey struct {
Value string `json:"value" required:"true"`
Enabled bool `json:"enabled" required:"true"`
Origin SpanMapperOrigin `json:"origin"`
}
// SpanMapperGroupCondition gates whether a group's rules run for a given span.
// A group runs when any attribute or resource key on the span CONTAINS one of
// the listed substrings (plain substring match — no glob syntax).
type SpanMapperGroupCondition struct {
Attributes []SpanMapperGroupConditionKey `json:"attributes" required:"true" nullable:"true"`
Resource []SpanMapperGroupConditionKey `json:"resource" required:"true" nullable:"true"`
Attributes []string `json:"attributes" required:"true" nullable:"true"`
Resource []string `json:"resource" required:"true" nullable:"true"`
}
// SpanMapperGroup is the domain model for a span attribute mapping group.
// Version is the shipped definition version for system groups and 0 otherwise.
type SpanMapperGroup struct {
types.TimeAuditable
types.UserAuditable
@@ -44,8 +31,6 @@ type SpanMapperGroup struct {
Name string `json:"name" required:"true"`
Condition SpanMapperGroupCondition `json:"condition" required:"true"`
Enabled bool `json:"enabled" required:"true"`
Origin SpanMapperOrigin `json:"origin" required:"true"`
Version int `json:"version" required:"true"`
}
// GettableSpanMapperGroup is the HTTP response representation of a mapping group.
@@ -73,42 +58,14 @@ type GettableSpanMapperGroups struct {
Items []*GettableSpanMapperGroup `json:"items" required:"true" nullable:"false"`
}
// Validate requires at least one substring overall and rejects blank ones.
// All-off is allowed: a group with every substring disabled simply never runs.
func (c *SpanMapperGroupCondition) Validate() error {
if len(c.Attributes)+len(c.Resource) == 0 {
return errors.New(errors.TypeInvalidInput, ErrCodeMappingInvalidInput, "condition must list at least one attribute or resource substring")
}
for _, k := range slices.Concat(c.Attributes, c.Resource) {
if strings.TrimSpace(k.Value) == "" {
return errors.New(errors.TypeInvalidInput, ErrCodeMappingInvalidInput, "condition substrings must not be blank")
}
if !k.Origin.IsZero() && k.Origin != SpanMapperOriginUser && k.Origin != SpanMapperOriginSystem {
return errors.Newf(errors.TypeInvalidInput, ErrCodeMappingInvalidInput, "condition origin must be one of %q or %q, got %q", SpanMapperOriginUser, SpanMapperOriginSystem, k.Origin.StringValue())
}
}
return nil
}
func (p *PostableSpanMapperGroup) Validate() error {
if strings.TrimSpace(p.Name) == "" {
return errors.New(errors.TypeInvalidInput, ErrCodeMappingInvalidInput, "group name must not be blank")
}
return p.Condition.Validate()
}
func NewSpanMapperGroup(orgID valuer.UUID, createdBy string, p *PostableSpanMapperGroup) *SpanMapperGroup {
now := time.Now()
return &SpanMapperGroup{
ID: valuer.GenerateUUID(),
OrgID: orgID,
Name: p.Name,
Condition: SpanMapperGroupCondition{
Attributes: conditionKeysWithOrigin(p.Condition.Attributes, SpanMapperOriginUser),
Resource: conditionKeysWithOrigin(p.Condition.Resource, SpanMapperOriginUser),
},
Enabled: p.Enabled,
Origin: SpanMapperOriginUser,
ID: valuer.GenerateUUID(),
OrgID: orgID,
Name: p.Name,
Condition: p.Condition,
Enabled: p.Enabled,
TimeAuditable: types.TimeAuditable{
CreatedAt: now,
UpdatedAt: now,
@@ -120,45 +77,18 @@ func NewSpanMapperGroup(orgID valuer.UUID, createdBy string, p *PostableSpanMapp
}
}
// Update applies a user edit. A system group keeps its name and its system
// substrings; see nextConditionKeys.
func (g *SpanMapperGroup) Update(name *string, condition *SpanMapperGroupCondition, enabled *bool, updatedBy string) error {
func (g *SpanMapperGroup) Update(name *string, condition *SpanMapperGroupCondition, enabled *bool, updatedBy string) {
if name != nil {
if g.Origin == SpanMapperOriginSystem && *name != g.Name {
return errors.Newf(errors.TypeInvalidInput, ErrCodeMappingInvalidInput, "system group %q cannot be renamed", g.Name)
}
if strings.TrimSpace(*name) == "" {
return errors.New(errors.TypeInvalidInput, ErrCodeMappingInvalidInput, "group name must not be blank")
}
g.Name = *name
}
if condition != nil {
attrs, err := nextConditionKeys(g.Condition.Attributes, condition.Attributes)
if err != nil {
return err
}
res, err := nextConditionKeys(g.Condition.Resource, condition.Resource)
if err != nil {
return err
}
g.Condition = SpanMapperGroupCondition{Attributes: attrs, Resource: res}
if err := g.Condition.Validate(); err != nil {
return err
}
g.Condition = *condition
}
if enabled != nil {
g.Enabled = *enabled
}
g.UpdatedAt = time.Now()
g.UpdatedBy = updatedBy
return nil
}
func (g *SpanMapperGroup) ErrIfNotDeletable() error {
if g.Origin == SpanMapperOriginSystem {
return errors.Newf(errors.TypeInvalidInput, ErrCodeMappingGroupNotDeletable, "system group %q cannot be deleted, disable it instead", g.Name)
}
return nil
}
func (g *SpanMapperGroup) ToStorable() *StorableSpanMapperGroup {
@@ -170,8 +100,6 @@ func (g *SpanMapperGroup) ToStorable() *StorableSpanMapperGroup {
Name: g.Name,
Condition: g.Condition,
Enabled: g.Enabled,
Origin: g.Origin,
Version: g.Version,
}
}
@@ -184,8 +112,6 @@ func (s *StorableSpanMapperGroup) ToSpanMapperGroup() *SpanMapperGroup {
Name: s.Name,
Condition: s.Condition,
Enabled: s.Enabled,
Origin: s.Origin,
Version: s.Version,
}
}
@@ -200,39 +126,3 @@ func NewSpanMapperGroupsFromStorable(ss []*StorableSpanMapperGroup) []*SpanMappe
func NewGettableSpanMapperGroups(g []*SpanMapperGroup) *GettableSpanMapperGroups {
return &GettableSpanMapperGroups{Items: g}
}
// nextConditionKeys builds a substring list from an edit: user substrings are
// taken from the edit as sent, system substrings stay as stored and the edit
// can only flip their enabled flag.
func nextConditionKeys(stored, edit []SpanMapperGroupConditionKey) ([]SpanMapperGroupConditionKey, error) {
var systemKeys, userKeys []SpanMapperGroupConditionKey
for _, k := range stored {
if k.Origin == SpanMapperOriginSystem {
systemKeys = append(systemKeys, k)
}
}
for _, k := range edit {
if k.Origin != SpanMapperOriginSystem {
k.Origin = SpanMapperOriginUser
userKeys = append(userKeys, k)
continue
}
idx := slices.IndexFunc(systemKeys, func(s SpanMapperGroupConditionKey) bool { return s.Value == k.Value })
if idx == -1 {
return nil, errors.Newf(errors.TypeInvalidInput, ErrCodeMappingInvalidInput, "system substring %q does not exist on this group; only its enabled flag can change", k.Value)
}
systemKeys[idx].Enabled = k.Enabled
}
return append(systemKeys, userKeys...), nil
}
func conditionKeysWithOrigin(keys []SpanMapperGroupConditionKey, origin SpanMapperOrigin) []SpanMapperGroupConditionKey {
out := make([]SpanMapperGroupConditionKey, len(keys))
for i, k := range keys {
k.Origin = origin
out[i] = k
}
return out
}

View File

@@ -1,121 +0,0 @@
package spantypes
import (
"bytes"
"encoding/json"
"slices"
"strings"
"github.com/SigNoz/signoz/pkg/errors"
)
var ErrCodeMappingDefinitionInvalid = errors.MustNewCode("span_attribute_mapping_definition_invalid")
// ProvisionerIdentity is stamped into created_by/updated_by by the reconciler.
const ProvisionerIdentity = "signoz"
// SpanMapperGroupDefinition is one shipped mapping group. Version is bumped on
// every content change and drives upgrades; the group name is the stable key
// and never changes. Once parsed, every substring and source carries the
// system origin and is enabled.
type SpanMapperGroupDefinition struct {
Version int `json:"version"`
Definition PostableSpanMapperTestGroup `json:"definition"`
}
func (d SpanMapperGroupDefinition) Name() string {
return d.Definition.Name
}
func NewSpanMapperGroupDefinition(raw []byte) (SpanMapperGroupDefinition, error) {
decoder := json.NewDecoder(bytes.NewReader(raw))
decoder.DisallowUnknownFields()
var d SpanMapperGroupDefinition
if err := decoder.Decode(&d); err != nil {
return SpanMapperGroupDefinition{}, errors.WrapInvalidInputf(err, ErrCodeMappingDefinitionInvalid, "%s", err.Error())
}
if err := d.validate(); err != nil {
return SpanMapperGroupDefinition{}, err
}
for _, keys := range [][]SpanMapperGroupConditionKey{d.Definition.Condition.Attributes, d.Definition.Condition.Resource} {
for i := range keys {
keys[i].Enabled = true
keys[i].Origin = SpanMapperOriginSystem
}
}
for i := range d.Definition.Mappers {
sources := d.Definition.Mappers[i].Config.Sources
for j := range sources {
sources[j].Enabled = true
sources[j].Origin = SpanMapperOriginSystem
}
}
return d, nil
}
// SpanMapperGroupRegistry holds every definition embedded in the binary, keyed by name.
type SpanMapperGroupRegistry struct {
definitions map[string]SpanMapperGroupDefinition
}
func NewSpanMapperGroupRegistry(definitions []SpanMapperGroupDefinition) (SpanMapperGroupRegistry, error) {
byName := make(map[string]SpanMapperGroupDefinition, len(definitions))
for _, d := range definitions {
if _, dup := byName[d.Name()]; dup {
return SpanMapperGroupRegistry{}, errors.NewInvalidInputf(ErrCodeMappingDefinitionInvalid, "duplicate span mapper group name %q", d.Name())
}
byName[d.Name()] = d
}
return SpanMapperGroupRegistry{definitions: byName}, nil
}
func (r SpanMapperGroupRegistry) IsReserved(name string) bool {
_, ok := r.definitions[name]
return ok
}
// List returns the definitions sorted by name so provisioning order is stable.
func (r SpanMapperGroupRegistry) List() []SpanMapperGroupDefinition {
out := make([]SpanMapperGroupDefinition, 0, len(r.definitions))
for _, d := range r.definitions {
out = append(out, d)
}
slices.SortFunc(out, func(a, b SpanMapperGroupDefinition) int { return strings.Compare(a.Name(), b.Name()) })
return out
}
func (d SpanMapperGroupDefinition) validate() error {
if d.Version < 1 {
return errors.NewInvalidInputf(ErrCodeMappingDefinitionInvalid, "version must be at least 1, got %d", d.Version)
}
if err := d.Definition.Validate(); err != nil {
return errors.Wrapf(err, errors.TypeInvalidInput, ErrCodeMappingDefinitionInvalid, "%s", d.Name())
}
if len(d.Definition.Mappers) == 0 {
return errors.NewInvalidInputf(ErrCodeMappingDefinitionInvalid, "%s: at least one mapper is required", d.Name())
}
names := make(map[string]struct{}, len(d.Definition.Mappers))
for i := range d.Definition.Mappers {
m := &d.Definition.Mappers[i]
if err := m.Validate(); err != nil {
return errors.Wrapf(err, errors.TypeInvalidInput, ErrCodeMappingDefinitionInvalid, "%s: mapper %q", d.Name(), m.Name)
}
if _, dup := names[m.Name]; dup {
return errors.NewInvalidInputf(ErrCodeMappingDefinitionInvalid, "%s: duplicate mapper %q", d.Name(), m.Name)
}
names[m.Name] = struct{}{}
for _, s := range m.Config.Sources {
if !s.Origin.IsZero() || s.Enabled {
return errors.NewInvalidInputf(ErrCodeMappingDefinitionInvalid, "%s: mapper %q: sources must not set origin or enabled", d.Name(), m.Name)
}
}
}
for _, k := range slices.Concat(d.Definition.Condition.Attributes, d.Definition.Condition.Resource) {
if !k.Origin.IsZero() || k.Enabled {
return errors.NewInvalidInputf(ErrCodeMappingDefinitionInvalid, "%s: condition substrings must not set origin or enabled", d.Name())
}
}
return nil
}

View File

@@ -86,31 +86,17 @@ func buildProcessorConfig(groups []*SpanMapperGroupWithMappers) *spanMapperProce
out := make([]spanMapperProcessorGroup, 0, len(groups))
for _, gm := range groups {
existsAny := spanMapperProcessorExistsAny{
Attributes: enabledConditionValues(gm.Group.Condition.Attributes),
Resource: enabledConditionValues(gm.Group.Condition.Resource),
}
// The collector rejects an empty exists_any and empty sources; with
// per-item toggles, all-off is valid stored state and means "never runs".
if len(existsAny.Attributes)+len(existsAny.Resource) == 0 {
continue
}
rules := make([]spanMapperProcessorAttribute, 0, len(gm.Mappers))
for _, m := range gm.Mappers {
rule := buildAttributeRule(m)
if len(rule.Sources) == 0 {
continue
}
rules = append(rules, rule)
}
if len(rules) == 0 {
continue
rules = append(rules, buildAttributeRule(m))
}
out = append(out, spanMapperProcessorGroup{
ID: gm.Group.Name,
ExistsAny: existsAny,
ID: gm.Group.Name,
ExistsAny: spanMapperProcessorExistsAny{
Attributes: gm.Group.Condition.Attributes,
Resource: gm.Group.Condition.Resource,
},
Attributes: rules,
})
}
@@ -118,31 +104,14 @@ func buildProcessorConfig(groups []*SpanMapperGroupWithMappers) *spanMapperProce
return &spanMapperProcessorConfig{Groups: out}
}
func enabledConditionValues(keys []SpanMapperGroupConditionKey) []string {
out := make([]string, 0, len(keys))
for _, k := range keys {
if k.Enabled {
out = append(out, k.Value)
}
}
if len(out) == 0 {
return nil
}
return out
}
// buildAttributeRule maps a single SpanMapper to a collector attribute rule.
// Disabled sources are skipped and the rest are sorted by Priority DESC
// (highest-priority first); read-from-resource sources are encoded via the
// "resource." prefix on the key. Each source carries its own action — "copy"
// is omitted to keep the emitted YAML compact, and only "move" is set explicitly.
// Sources are sorted by Priority DESC (highest-priority first); read-from-
// resource sources are encoded via the "resource." prefix on the key. Each
// source carries its own action — "copy" is omitted to keep the emitted YAML
// compact, and only "move" is set explicitly.
func buildAttributeRule(m *SpanMapper) spanMapperProcessorAttribute {
sources := make([]SpanMapperSource, 0, len(m.Config.Sources))
for _, s := range m.Config.Sources {
if s.Enabled {
sources = append(sources, s)
}
}
sources := make([]SpanMapperSource, len(m.Config.Sources))
copy(sources, m.Config.Sources)
sort.SliceStable(sources, func(i, j int) bool { return sources[i].Priority > sources[j].Priority })
out := make([]spanMapperProcessorSource, 0, len(sources))

View File

@@ -145,22 +145,6 @@ func TestBuildAttributeRule(t *testing.T) {
},
},
},
{
name: "disabled_sources_skipped",
mapper: newMapper("gen_ai.input.messages", FieldContextSpanAttribute,
systemSrc("gen_ai.prompt", SpanMapperOperationCopy, 30, false),
systemSrc("input.value", SpanMapperOperationCopy, 20, true),
attrSrc("gen_ai.prompt", SpanMapperOperationMove, 40),
),
want: spanMapperProcessorAttribute{
Target: "gen_ai.input.messages",
Context: FieldContextSpanAttribute.StringValue(),
Sources: []spanMapperProcessorSource{
{Key: "gen_ai.prompt", Action: SpanMapperOperationMove.StringValue()},
{Key: "input.value"},
},
},
},
}
for _, tc := range tests {
@@ -171,33 +155,6 @@ func TestBuildAttributeRule(t *testing.T) {
}
}
func TestBuildProcessorConfigDropsAllOffItems(t *testing.T) {
t.Parallel()
offGroup := newGroup("all-off", nil, nil)
offGroup.Condition.Attributes = []SpanMapperGroupConditionKey{{Value: "model", Enabled: false, Origin: SpanMapperOriginSystem}}
mixed := newGroup("llm", nil, nil)
mixed.Condition.Attributes = []SpanMapperGroupConditionKey{
{Value: "model", Enabled: false, Origin: SpanMapperOriginSystem},
{Value: "gen_ai.request.model", Enabled: true, Origin: SpanMapperOriginUser},
}
got := buildProcessorConfig([]*SpanMapperGroupWithMappers{
{Group: offGroup, Mappers: []*SpanMapper{newMapper("gen_ai.request.model", FieldContextSpanAttribute, attrSrc("llm.model", SpanMapperOperationCopy, 1))}},
{Group: mixed, Mappers: []*SpanMapper{
newMapper("gen_ai.request.model", FieldContextSpanAttribute, systemSrc("llm.model", SpanMapperOperationCopy, 10, false)),
newMapper("gen_ai.provider.name", FieldContextSpanAttribute, systemSrc("llm.vendor", SpanMapperOperationCopy, 10, true)),
}},
})
require.Len(t, got.Groups, 1)
assert.Equal(t, "llm", got.Groups[0].ID)
assert.Equal(t, []string{"gen_ai.request.model"}, got.Groups[0].ExistsAny.Attributes)
require.Len(t, got.Groups[0].Attributes, 1)
assert.Equal(t, "gen_ai.provider.name", got.Groups[0].Attributes[0].Target)
}
func loadFixture(t *testing.T, name string) []byte {
t.Helper()
b, err := os.ReadFile(filepath.Join("testdata", name))
@@ -217,23 +174,12 @@ func assertYAMLEqual(t *testing.T, want, got []byte) {
func newGroup(name string, attrs, res []string) *SpanMapperGroup {
return &SpanMapperGroup{
Name: name,
Condition: SpanMapperGroupCondition{
Attributes: userConditionKeys(attrs),
Resource: userConditionKeys(res),
},
Enabled: true,
Name: name,
Condition: SpanMapperGroupCondition{Attributes: attrs, Resource: res},
Enabled: true,
}
}
func userConditionKeys(values []string) []SpanMapperGroupConditionKey {
out := make([]SpanMapperGroupConditionKey, len(values))
for i, v := range values {
out[i] = SpanMapperGroupConditionKey{Value: v, Enabled: true, Origin: SpanMapperOriginUser}
}
return out
}
func newMapper(name string, target FieldContext, sources ...SpanMapperSource) *SpanMapper {
return &SpanMapper{
Name: name,
@@ -244,13 +190,9 @@ func newMapper(name string, target FieldContext, sources ...SpanMapperSource) *S
}
func attrSrc(key string, op SpanMapperOperation, priority int) SpanMapperSource {
return SpanMapperSource{Key: key, Context: FieldContextSpanAttribute, Operation: op, Priority: priority, Enabled: true, Origin: SpanMapperOriginUser}
return SpanMapperSource{Key: key, Context: FieldContextSpanAttribute, Operation: op, Priority: priority}
}
func resSrc(key string, op SpanMapperOperation, priority int) SpanMapperSource {
return SpanMapperSource{Key: key, Context: FieldContextResource, Operation: op, Priority: priority, Enabled: true, Origin: SpanMapperOriginUser}
}
func systemSrc(key string, op SpanMapperOperation, priority int, enabled bool) SpanMapperSource {
return SpanMapperSource{Key: key, Context: FieldContextSpanAttribute, Operation: op, Priority: priority, Enabled: enabled, Origin: SpanMapperOriginSystem}
return SpanMapperSource{Key: key, Context: FieldContextResource, Operation: op, Priority: priority}
}

View File

@@ -15,14 +15,14 @@ func TestSimulateSpanMappersProcessing_EndToEnd(t *testing.T) {
groups := []*SpanMapperGroupWithMappers{{
Group: &SpanMapperGroup{
Name: "llm",
Condition: SpanMapperGroupCondition{Attributes: userConditionKeys([]string{"model"})},
Condition: SpanMapperGroupCondition{Attributes: []string{"model"}},
Enabled: true,
},
Mappers: []*SpanMapper{{
Name: "gen_ai.request.model",
FieldContext: FieldContextSpanAttribute,
Config: SpanMapperConfig{Sources: []SpanMapperSource{
{Key: "llm.model", Context: FieldContextSpanAttribute, Operation: SpanMapperOperationCopy, Priority: 1, Enabled: true, Origin: SpanMapperOriginUser},
{Key: "llm.model", Context: FieldContextSpanAttribute, Operation: SpanMapperOperationCopy, Priority: 1},
}},
Enabled: true,
}},

View File

@@ -20,9 +20,7 @@ type StorableSpanMapperGroup struct {
OrgID valuer.UUID `bun:"org_id,type:text,notnull"`
Name string `bun:"name,type:text,notnull"`
Condition SpanMapperGroupCondition `bun:"condition,type:jsonb,notnull"`
Enabled bool `bun:"enabled,notnull"`
Origin SpanMapperOrigin `bun:"origin,type:text,notnull"`
Version int `bun:"version,notnull"`
Enabled bool `bun:"enabled,notnull,default:true"`
}
type StorableSpanMapper struct {
@@ -36,8 +34,7 @@ type StorableSpanMapper struct {
Name string `bun:"name,type:text,notnull"`
FieldContext FieldContext `bun:"field_context,type:text,notnull"`
Config SpanMapperConfig `bun:"config,type:jsonb,notnull"`
Enabled bool `bun:"enabled,notnull"`
Origin SpanMapperOrigin `bun:"origin,type:text,notnull"`
Enabled bool `bun:"enabled,notnull,default:true"`
}
func (c SpanMapperGroupCondition) Value() (driver.Value, error) {

View File

@@ -9,14 +9,9 @@ import (
)
type SpanMapperStore interface {
// RunInTx runs cb in one transaction; every store call made with the
// callback's ctx joins it.
RunInTx(ctx context.Context, cb func(ctx context.Context) error) error
// Group operations
ListGroups(ctx context.Context, orgID valuer.UUID, q *ListSpanMapperGroupsQuery) ([]*SpanMapperGroup, error)
GetGroup(ctx context.Context, orgID, id valuer.UUID) (*SpanMapperGroup, error)
GetGroupByName(ctx context.Context, orgID valuer.UUID, name string) (*SpanMapperGroup, error)
CreateGroup(ctx context.Context, group *SpanMapperGroup) error
UpdateGroup(ctx context.Context, group *SpanMapperGroup) error
DeleteGroup(ctx context.Context, orgID, id valuer.UUID) error

View File

@@ -10,6 +10,7 @@ from fixtures import types
from fixtures.auth import (
USER_ADMIN_EMAIL,
USER_ADMIN_PASSWORD,
create_active_user,
)
TIMEOUT = 10

View File

@@ -41,7 +41,7 @@ def test_create_groups_and_simulate_with_backfill(
},
json={
"name": "llm-backfill",
"condition": {"attributes": [{"value": "model", "enabled": True}], "resource": []},
"condition": {"attributes": ["model"], "resource": []},
"enabled": True,
},
)
@@ -69,7 +69,6 @@ def test_create_groups_and_simulate_with_backfill(
"context": "attribute",
"operation": "copy",
"priority": 1,
"enabled": True,
}
]
},
@@ -127,13 +126,13 @@ def test_create_groups_and_simulate_with_backfill(
# No "mappers" key: the server backfills them from the saved group.
{
"name": "llm-backfill",
"condition": {"attributes": [{"value": "model", "enabled": True}], "resource": []},
"condition": {"attributes": ["model"], "resource": []},
"enabled": True,
},
# Unsaved group; mappers provided inline.
{
"name": "db-inline",
"condition": {"attributes": [{"value": "db", "enabled": True}], "resource": []},
"condition": {"attributes": ["db"], "resource": []},
"enabled": True,
"mappers": [
{
@@ -146,7 +145,6 @@ def test_create_groups_and_simulate_with_backfill(
"context": "attribute",
"operation": "move",
"priority": 1,
"enabled": True,
}
]
},

View File

@@ -1,152 +0,0 @@
from collections.abc import Callable
from http import HTTPStatus
import requests
from fixtures import types
from fixtures.auth import USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD
GROUPS_PATH = "/api/v1/span_mapper_groups"
def test_default_groups_are_seeded_and_shipped_items_are_toggle_only(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
) -> None:
"""
Setup:
A fresh org. The reconciler seeds the shipped mapping groups at startup
and on org creation, so nothing has to be created here.
Tests:
1. The list contains llm, agent and tool as system groups with shipped
substrings
2. Shipped mappers and their sources are system-owned and enabled
3. A shipped name cannot be taken by a user group, and system groups and
mappers cannot be deleted
4. A shipped source can be switched off and a user override added; both
round-trip through PATCH and the simulator honours them
5. A shipped substring can be switched off and a user one added
"""
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
headers = {"authorization": f"Bearer {token}", "content-type": "application/json"}
list_groups = requests.get(signoz.self.host_configs["8080"].get(GROUPS_PATH), timeout=10, headers=headers)
assert list_groups.status_code == HTTPStatus.OK
groups = {g["name"]: g for g in list_groups.json()["data"]["items"]}
assert {"llm", "agent", "tool"} <= set(groups)
for name in ("llm", "agent", "tool"):
assert groups[name]["origin"] == "system"
assert groups[name]["version"] >= 1
assert groups[name]["createdBy"] == "signoz"
llm = groups["llm"]
assert llm["condition"]["attributes"] == [{"value": "model", "enabled": True, "origin": "system"}]
list_mappers = requests.get(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{llm['id']}/span_mappers"), timeout=10, headers=headers)
assert list_mappers.status_code == HTTPStatus.OK
mappers = {m["name"]: m for m in list_mappers.json()["data"]["items"]}
model = mappers["gen_ai.request.model"]
assert model["origin"] == "system"
assert model["enabled"] is True
assert all(s["origin"] == "system" and s["enabled"] is True for s in model["config"]["sources"])
assert "llm.model_name" in [s["key"] for s in model["config"]["sources"]]
reserved = requests.post(
signoz.self.host_configs["8080"].get(GROUPS_PATH),
timeout=10,
headers=headers,
json={"name": "tool", "condition": {"attributes": [{"value": "tool", "enabled": True}], "resource": []}, "enabled": True},
)
assert reserved.status_code == HTTPStatus.BAD_REQUEST
assert reserved.json()["error"]["code"] == "span_attribute_mapping_group_name_reserved"
delete_group = requests.delete(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{llm['id']}"), timeout=10, headers=headers)
assert delete_group.status_code == HTTPStatus.BAD_REQUEST
assert delete_group.json()["error"]["code"] == "span_attribute_mapping_group_not_deletable"
delete_mapper = requests.delete(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{llm['id']}/span_mappers/{model['id']}"), timeout=10, headers=headers)
assert delete_mapper.status_code == HTTPStatus.BAD_REQUEST
assert delete_mapper.json()["error"]["code"] == "span_attribute_mapper_not_deletable"
# Switch the shipped llm.model_name source off and re-add it as a user move.
sources = [{**s, "enabled": s["key"] != "llm.model_name"} for s in model["config"]["sources"]] + [{"key": "llm.model_name", "context": "attribute", "operation": "move", "priority": 1, "enabled": True}]
patch_mapper = requests.patch(
signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{llm['id']}/span_mappers/{model['id']}"),
timeout=10,
headers=headers,
json={"config": {"sources": sources}},
)
assert patch_mapper.status_code == HTTPStatus.NO_CONTENT
list_mappers = requests.get(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{llm['id']}/span_mappers"), timeout=10, headers=headers)
updated = {m["name"]: m for m in list_mappers.json()["data"]["items"]}["gen_ai.request.model"]
by_origin = {(s["key"], s["origin"]): s for s in updated["config"]["sources"]}
assert by_origin[("llm.model_name", "system")]["enabled"] is False
assert by_origin[("llm.model_name", "user")]["operation"] == "move"
assert len(updated["config"]["sources"]) == len(model["config"]["sources"]) + 1
# The user override wins: the source is moved, not copied.
simulate = requests.post(
signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/test"),
timeout=10,
headers=headers,
json={
"spans": [{"attributes": {"llm.model_name": "gpt-4o"}, "resource": {}}],
"groups": [{"name": "llm", "condition": llm["condition"], "enabled": True}],
},
)
assert simulate.status_code == HTTPStatus.OK
attrs = simulate.json()["data"]["spans"][0]["attributes"]
assert attrs["gen_ai.request.model"] == "gpt-4o"
assert "llm.model_name" not in attrs
# A shipped substring that does not exist is rejected; toggling one is not.
bad_condition = requests.patch(
signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{llm['id']}"),
timeout=10,
headers=headers,
json={"condition": {"attributes": [{"value": "nope", "enabled": True, "origin": "system"}], "resource": []}},
)
assert bad_condition.status_code == HTTPStatus.BAD_REQUEST
patch_group = requests.patch(
signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{llm['id']}"),
timeout=10,
headers=headers,
json={
"condition": {
"attributes": [
{"value": "model", "enabled": False, "origin": "system"},
{"value": "gen_ai.request.model", "enabled": True},
],
"resource": [],
}
},
)
assert patch_group.status_code == HTTPStatus.NO_CONTENT
list_groups = requests.get(signoz.self.host_configs["8080"].get(GROUPS_PATH), timeout=10, headers=headers)
llm_after = {g["name"]: g for g in list_groups.json()["data"]["items"]}["llm"]
assert llm_after["condition"]["attributes"] == [
{"value": "model", "enabled": False, "origin": "system"},
{"value": "gen_ai.request.model", "enabled": True, "origin": "user"},
]
assert llm_after["origin"] == "system"
assert llm_after["name"] == "llm"
# Leave the shipped group as seeded for the other suites.
restore_group = requests.patch(
signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{llm['id']}"),
timeout=10,
headers=headers,
json={"condition": {"attributes": [{"value": "model", "enabled": True, "origin": "system"}], "resource": []}},
)
assert restore_group.status_code == HTTPStatus.NO_CONTENT
restore_mapper = requests.patch(
signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{llm['id']}/span_mappers/{model['id']}"),
timeout=10,
headers=headers,
json={"config": {"sources": model["config"]["sources"]}},
)
assert restore_mapper.status_code == HTTPStatus.NO_CONTENT