Compare commits

..

4 Commits

Author SHA1 Message Date
Srikanth Chekuri
95fc905448 chore: move savedview and integration types into their own types package (#10454)
Some checks are pending
build-staging / staging (push) Blocked by required conditions
build-staging / prepare (push) Waiting to run
build-staging / js-build (push) Blocked by required conditions
build-staging / go-build (push) Blocked by required conditions
Release Drafter / update_release_draft (push) Waiting to run
2026-02-28 12:55:26 +00:00
Vinicius Lourenço
9f5afaf36f perf(bundle-size): be explicit when including new languages for react-syntax-highlighter (#10228)
Some checks failed
build-staging / staging (push) Has been cancelled
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
2026-02-28 14:48:50 +05:30
Srikanth Chekuri
3f69d5cdc2 chore: add docs for service (#10450) 2026-02-28 13:59:42 +05:30
Srikanth Chekuri
98617b5120 chore: add doc for adding new abstractions to codebase (#10444)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
* chore: add doc for adding new abstractions to codebase

* chore: reorder

---------

Co-authored-by: Pandey <vibhupandey28@gmail.com>
2026-02-27 20:55:15 +00:00
28 changed files with 605 additions and 1232 deletions

View File

@@ -61,3 +61,13 @@ jobs:
with:
PRIMUS_REF: main
JS_SRC: frontend
md-languages:
if: |
(github.event_name == 'pull_request' && ! github.event.pull_request.head.repo.fork && github.event.pull_request.user.login != 'dependabot[bot]' && ! contains(github.event.pull_request.labels.*.name, 'safe-to-test')) ||
(github.event_name == 'pull_request_target' && contains(github.event.pull_request.labels.*.name, 'safe-to-test'))
runs-on: ubuntu-latest
steps:
- name: checkout
uses: actions/checkout@v4
- name: validate md languages
run: bash frontend/scripts/validate-md-languages.sh

View File

@@ -0,0 +1,127 @@
# Abstractions
This document provides rules for deciding when a new type, interface, or intermediate representation is warranted in Go code. The goal is to keep the codebase navigable by ensuring every abstraction earns its place.
## The cost of a new abstraction
Every exported type, interface, or wrapper is a permanent commitment. It must be named, documented, tested, and understood by every future contributor. It creates a new concept in the codebase vocabulary. Before introducing one, verify that the cost is justified by a concrete benefit that cannot be achieved with existing mechanisms.
## Before you introduce anything new
Answer these four questions. If writing a PR, include the answers in the description.
1. **What already exists?** Name the specific type, function, interface, or library that covers this ground today.
2. **What does the new abstraction add?** Name the concrete operation, guarantee, or capability. "Cleaner" or "more reusable" are not sufficient; name what the caller can do that it could not do before.
3. **What does the new abstraction drop?** If it wraps or mirrors an existing structure, list what it cannot represent. Every gap must be either justified or handled with an explicit error.
4. **Who consumes it?** List the call sites. If there is only one producer and one consumer in the same call chain, you likely need a function, not a type.
## Rules
### 1. Prefer functions over types
If a piece of logic has one input and one output, write a function. Do not create a struct to hold intermediate state that is built in one place and read in one place. A function is easier to test, easier to inline, and does not expand the vocabulary of the codebase.
```go
// Prefer this:
func ConvertConfig(src ExternalConfig) (InternalConfig, error)
// Over this:
type ConfigAdapter struct { ... }
func NewConfigAdapter(src ExternalConfig) *ConfigAdapter
func (a *ConfigAdapter) ToInternal() (InternalConfig, error)
```
The two-step version is only justified when `ConfigAdapter` has multiple distinct consumers that use it in different ways.
### 2. Do not duplicate structures you do not own
When a library or external package produces a structured output, operate on that output directly. Do not create a parallel type that mirrors a subset of its fields.
A partial copy will:
- **Silently lose data** when the source has fields or variants the copy does not account for.
- **Drift** when the source evolves and the copy is not updated in lockstep.
- **Add a conversion step** that doubles the code surface and the opportunity for bugs.
If you need to shield consumers from a dependency, define a narrow interface over the dependency's type rather than copying its shape into a new struct.
### 3. Never silently discard input
If your code receives structured input and cannot handle part of it, return an error. Do not silently return nil, skip the element, or produce a partial result. Silent data loss is the hardest class of bug to detect because the code appears to work, it just produces wrong results.
```go
// Wrong: silently ignores the unrecognized case.
default:
return nil
// Right: makes the gap visible.
default:
return nil, fmt.Errorf("unsupported %T value: %v", v, v)
```
This applies broadly: type switches, format conversions, data migrations, enum mappings, configuration parsing. Anywhere a `default` or `else` branch can swallow input, it should surface an error instead.
### 4. Do not expose methods that lose information
A method on a structured type should not strip meaning from the structure it belongs to. If a caller needs to iterate over elements for a specific purpose (validation, aggregation, logging), write that logic as a standalone function that operates on the structure with full context, rather than adding a method that returns a reduced view.
```go
// Problematic: callers cannot distinguish how items were related.
func (o *Order) AllLineItems() []LineItem { ... }
// Better: the validation logic operates on the full structure.
func ValidateOrder(o *Order) error { ... }
```
Public methods shape how a type is used. Once a lossy accessor exists, callers will depend on it, and the lost information becomes unrecoverable at those call sites.
### 5. Interfaces should be discovered, not predicted
Do not define an interface before you have at least two concrete implementations that need it. An interface with one implementation is not abstraction; it is indirection that makes it harder to navigate from call site to implementation.
The exception is interfaces required for testing (e.g., for mocking an external dependency). In that case, define the interface in the **consuming** package, not the providing package, following the Go convention of [accepting interfaces and returning structs](https://go.dev/wiki/CodeReviewComments#interfaces).
### 6. Wrappers must add semantics, not just rename
A wrapper type is justified when it adds meaning, validation, or invariants that the underlying type does not carry. It is not justified when it merely renames fields or reorganizes the same data into a different shape.
```go
// Justified: adds validation that the underlying string does not carry.
type OrgID struct{ value string }
func NewOrgID(s string) (OrgID, error) { /* validates format */ }
// Not justified: renames fields with no new invariant or behavior.
type UserInfo struct {
Name string // same as source.Name
Email string // same as source.Email
}
```
Ask: what does the wrapper guarantee that the underlying type does not? If the answer is nothing, use the underlying type directly.
## When a new type IS warranted
A new type earns its place when it meets **at least one** of these criteria:
- **Serialization boundary**: It must be persisted, sent over the wire, or written to config. The source type is unsuitable (unexported fields, function pointers, cycles).
- **Invariant enforcement**: The constructor or methods enforce constraints that raw data does not carry (e.g., non-empty, validated format, bounded range).
- **Multiple distinct consumers**: Three or more call sites use the type in meaningfully different ways. The type is the shared vocabulary between them.
- **Dependency firewall**: The type lives in a lightweight package so that consumers avoid importing a heavy dependency.
## What should I remember?
- A function is almost always simpler than a type. Start with a function; promote to a type only when you have evidence of need.
- Never silently drop data. If you cannot handle it, error.
- If your new type mirrors an existing one, you need a strong reason beyond "nicer to work with".
- If your type has one producer and one consumer, it is indirection, not abstraction.
- Interfaces come from need (multiple implementations), not from prediction.
- When in doubt, do not add it. It is easier to add an abstraction later when the need is clear than to remove one after it has spread through the codebase.
## Further reading
These works and our own lessions shaped the above guidelines
- [The Wrong Abstraction](https://sandimetz.com/blog/2016/1/20/the-wrong-abstraction) - Sandi Metz. The wrong abstraction is worse than duplication. If you find yourself passing parameters and adding conditional paths through shared code, inline it back into every caller and let the duplication show you what the right abstraction is.
- [Write code that is easy to delete, not easy to extend](https://programmingisterrible.com/post/139222674273/write-code-that-is-easy-to-delete-not-easy-to) - tef. Every abstraction is a bet on the future. Optimize for how cheaply you can remove code when the bet is wrong, not for how easily you can extend it when the bet is right.
- [Goodbye, Clean Code](https://overreacted.io/goodbye-clean-code/) - Dan Abramov. A refactoring that removes duplication can look cleaner while making the code harder to change. Clean-looking and easy-to-change are not the same thing.
- [A Philosophy of Software Design](https://www.amazon.com/Philosophy-Software-Design-John-Ousterhout/dp/1732102201) - John Ousterhout. Good abstractions are deep: simple interface, complex implementation. A "false abstraction" omits important details while appearing simple, and is worse than no abstraction at all. ([Summary by Pragmatic Engineer](https://blog.pragmaticengineer.com/a-philosophy-of-software-design-review/))
- [Simplicity is Complicated](https://go.dev/talks/2015/simplicity-is-complicated.slide) - Rob Pike. Go-specific. Fewer orthogonal concepts that compose predictably beat many overlapping ones. Features were left out of Go deliberately; the same discipline applies to your own code.

View File

@@ -10,11 +10,13 @@ We adhere to three primary style guides as our foundation:
We **recommend** (almost enforce) reviewing these guides before contributing to the codebase. They provide valuable insights into writing idiomatic Go code and will help you understand our approach to backend development. In addition, we have a few additional rules that make certain areas stricter than the above which can be found in area-specific files in this package:
- [Packages](packages.md) — Naming, layout, and conventions for `pkg/` packages
- [Errors](errors.md) Structured error handling
- [Handler](handler.md) — Writing HTTP handlers and OpenAPI integration
- [Endpoint](endpoint.md) — Endpoint conventions
- [SQL](sql.md) — Database query patterns
- [Provider](provider.md) — Provider pattern
- [Integration](integration.md) — Integration conventions
- [Flagger](flagger.md) — Feature flag conventions
- [Abstractions](abstractions.md) - When to introduce new types and intermediate representations
- [Errors](errors.md) - Structured error handling
- [Endpoint](endpoint.md) - HTTP endpoint patterns
- [Flagger](flagger.md) - Feature flag patterns
- [Handler](handler.md) - HTTP handler patterns
- [Integration](integration.md) - Integration testing
- [Provider](provider.md) - Dependency injection and provider patterns
- [Packages](packages.md) - Naming, layout, and conventions for `pkg/` packages
- [Service](service.md) - Managed service lifecycle with `factory.Service`
- [SQL](sql.md) - Database and SQL patterns

View File

@@ -0,0 +1,269 @@
# Service
A service is a component with a managed lifecycle: it starts, runs for the lifetime of the application, and stops gracefully.
Services are distinct from [providers](provider.md). A provider adapts an external dependency behind an interface. A service has a managed lifecycle that is tied to the lifetime of the application.
## When do you need a service?
You need a service when your component needs to do work that outlives a single method call:
- **Periodic work**: polling an external system, garbage-collecting expired data, syncing state on an interval.
- **Graceful shutdown**: holding resources (connections, caches, buffers) that must be flushed or closed before the process exits.
- **Blocking on readiness**: waiting for an external dependency to become available before the application can proceed.
If your component only responds to calls and holds no state that requires cleanup, it is a provider, not a service. If it does both (responds to calls *and* needs a lifecycle), embed `factory.Service` in the provider interface; see [How to create a service](#how-to-create-a-service).
## The interface
The `factory.Service` interface in `pkg/factory/service.go` defines two methods:
```go
type Service interface {
// Starts a service. It should block and should not return until the service is stopped or it fails.
Start(context.Context) error
// Stops a service.
Stop(context.Context) error
}
```
`Start` **must block**. It should not return until the service is stopped (returning `nil`) or something goes wrong (returning an error). If `Start` returns an error, the entire application shuts down.
`Stop` should cause `Start` to unblock and return. It must be safe to call from a different goroutine than the one running `Start`.
## Shutdown coordination
Every service uses a `stopC chan struct{}` to coordinate shutdown:
- **Constructor**: `stopC: make(chan struct{})`
- **Start**: blocks on `<-stopC` (or uses it in a `select` loop)
- **Stop**: `close(stopC)` to unblock `Start`
This is the standard pattern. Do not use `context.WithCancel` or other mechanisms for service-level shutdown coordination. See the examples in the next section.
## Service shapes
Two shapes recur across the codebase (these are not exhaustive, if a new shape is needed, bring it up for discussion before going ahead with the implementation), implemented by convention rather than base classes.
### Idle service
The service does work during startup or shutdown but has nothing to do while running. `Start` blocks on `<-stopC`. `Stop` closes `stopC` and optionally does cleanup.
The JWT tokenizer (`pkg/tokenizer/jwttokenizer/provider.go`) is a good example. It validates and creates tokens on demand via method calls, but has no periodic work to do. It still needs the service lifecycle so the registry can manage its lifetime:
```go
// pkg/tokenizer/jwttokenizer/provider.go
func (provider *provider) Start(ctx context.Context) error {
<-provider.stopC
return nil
}
func (provider *provider) Stop(ctx context.Context) error {
close(provider.stopC)
return nil
}
```
The instrumentation SDK (`pkg/instrumentation/sdk.go`) is idle while running but does real cleanup in `Stop` shutting down its OpenTelemetry tracer and meter providers:
```go
// pkg/instrumentation/sdk.go
func (i *SDK) Start(ctx context.Context) error {
<-i.startCh
return nil
}
func (i *SDK) Stop(ctx context.Context) error {
close(i.startCh)
return errors.Join(
i.sdk.Shutdown(ctx),
i.meterProviderShutdownFunc(ctx),
)
}
```
### Scheduled service
The service runs an operation repeatedly on a fixed interval. `Start` runs a ticker loop with a `select` on `stopC` and the ticker channel.
The opaque tokenizer (`pkg/tokenizer/opaquetokenizer/provider.go`) garbage-collects expired tokens and flushes cached last-observed-at timestamps to the database on a configurable interval:
```go
// pkg/tokenizer/opaquetokenizer/provider.go
func (provider *provider) Start(ctx context.Context) error {
ticker := time.NewTicker(provider.config.Opaque.GC.Interval)
defer ticker.Stop()
for {
select {
case <-provider.stopC:
return nil
case <-ticker.C:
orgs, err := provider.orgGetter.ListByOwnedKeyRange(ctx)
if err != nil {
provider.settings.Logger().ErrorContext(ctx, "failed to get orgs data", "error", err)
continue
}
for _, org := range orgs {
if err := provider.gc(ctx, org); err != nil {
provider.settings.Logger().ErrorContext(ctx, "failed to garbage collect tokens", "error", err, "org_id", org.ID)
}
if err := provider.flushLastObservedAt(ctx, org); err != nil {
provider.settings.Logger().ErrorContext(ctx, "failed to flush tokens", "error", err, "org_id", org.ID)
}
}
}
}
}
```
Its `Stop` does a final gc and flush before returning, so no data is lost on shutdown:
```go
// pkg/tokenizer/opaquetokenizer/provider.go
func (provider *provider) Stop(ctx context.Context) error {
close(provider.stopC)
orgs, err := provider.orgGetter.ListByOwnedKeyRange(ctx)
if err != nil {
return err
}
for _, org := range orgs {
if err := provider.gc(ctx, org); err != nil {
provider.settings.Logger().ErrorContext(ctx, "failed to garbage collect tokens", "error", err, "org_id", org.ID)
}
if err := provider.flushLastObservedAt(ctx, org); err != nil {
provider.settings.Logger().ErrorContext(ctx, "failed to flush tokens", "error", err, "org_id", org.ID)
}
}
return nil
}
```
The key points:
- In the loop, `select` on `stopC` and the ticker. Errors in iterations are logged but do not cause the service to return (which would shut down the application).
- Only return an error from `Start` if the failure is unrecoverable.
- Use `Stop` to flush or drain any in-memory state before the process exits.
## How to create a service
There are two cases: a standalone service and a provider that is also a service.
### Standalone service
A standalone service only has the `factory.Service` lifecycle i.e it does not serve as a dependency for other packages. The user reconciliation service is an example.
1. Define the service interface in your package. Embed `factory.Service`:
```go
// pkg/modules/user/service.go
package user
type Service interface {
factory.Service
}
```
2. Create the implementation in an `impl` sub-package. Use an unexported struct with an exported constructor that returns the interface:
```go
// pkg/modules/user/impluser/service.go
package impluser
type service struct {
settings factory.ScopedProviderSettings
// ... dependencies ...
stopC chan struct{}
}
func NewService(
providerSettings factory.ProviderSettings,
// ... dependencies ...
) user.Service {
return &service{
settings: factory.NewScopedProviderSettings(providerSettings, "go.signoz.io/pkg/modules/user"),
// ... dependencies ...
stopC: make(chan struct{}),
}
}
func (s *service) Start(ctx context.Context) error { ... }
func (s *service) Stop(ctx context.Context) error { ... }
```
### Provider that is also a service
Many providers need a managed lifecycle: they poll, sync, or garbage-collect in the background. In this case, embed `factory.Service` in the provider interface. The implementation satisfies both the provider methods and `Start`/`Stop`.
```go
// pkg/tokenizer/tokenizer.go
package tokenizer
type Tokenizer interface {
factory.Service
CreateToken(context.Context, *authtypes.Identity, map[string]string) (*authtypes.Token, error)
GetIdentity(context.Context, string) (*authtypes.Identity, error)
// ... other methods ...
}
```
The implementation (e.g. `pkg/tokenizer/opaquetokenizer/provider.go`) implements `Start`, `Stop`, and all the provider methods on the same struct. See the [provider guide](provider.md) for how to set up the factory, config, and constructor. The `stopC` channel and `Start`/`Stop` methods follow the same patterns described above.
## How to wire it up
Wiring happens in `pkg/signoz/signoz.go`.
### 1. Instantiate the service
For a standalone service, call the constructor directly:
```go
userService := impluser.NewService(providerSettings, store, module, orgGetter, authz, config.User.Root)
```
For a provider that is also a service, use `factory.NewProviderFromNamedMap` as described in the [provider guide](provider.md). The returned value already implements `factory.Service`.
### 2. Register in the registry
Wrap the service with `factory.NewNamedService` and pass it to `factory.NewRegistry`:
```go
registry, err := factory.NewRegistry(
instrumentation.Logger(),
// ... other services ...
factory.NewNamedService(factory.MustNewName("user"), userService),
)
```
The name must be unique across all services. The registry handles the rest:
- **Start**: launches all services concurrently in goroutines.
- **Wait**: blocks until a service returns an error, the context is cancelled, or a SIGINT/SIGTERM is received. Any service error triggers application shutdown.
- **Stop**: stops all services concurrently, collects errors via `errors.Join`.
You do not call `Start` or `Stop` on individual services. The registry does it.
## What should I remember?
- A service has a managed lifecycle: `Start` blocks, `Stop` unblocks it.
- Use `stopC chan struct{}` for shutdown coordination. `close(stopC)` in `Stop`, `<-stopC` in `Start`.
- Service shapes: idle (block on `stopC`) and scheduled (ticker loop with `select`).
- Unexported struct, exported `NewService` constructor returning the interface.
- First constructor parameter is `factory.ProviderSettings`. Create scoped settings with `factory.NewScopedProviderSettings`.
- Register in `factory.Registry` with `factory.NewNamedService`. The registry starts and stops everything.
- Only return an error from `Start` if the failure is unrecoverable. Log and continue for transient errors in polling loops.
## Further reading
- [Google Guava - ServiceExplained](https://github.com/google/guava/wiki/ServiceExplained) - the service lifecycle pattern takes inspiration from this
- [OpenTelemetry Collector](https://github.com/open-telemetry/opentelemetry-collector) - Worth studying for its approach to building composable components

View File

@@ -19,6 +19,8 @@ const config: Config.InitialOptions = {
'^.*/useSafeNavigate$': USE_SAFE_NAVIGATE_MOCK_PATH,
'^@signozhq/icons$':
'<rootDir>/node_modules/@signozhq/icons/dist/index.esm.js',
'^react-syntax-highlighter/dist/esm/(.*)$':
'<rootDir>/node_modules/react-syntax-highlighter/dist/cjs/$1',
},
globals: {
extensionsToTreatAsEsm: ['.ts'],

View File

@@ -214,7 +214,7 @@
"@types/react-redux": "^7.1.11",
"@types/react-resizable": "3.0.3",
"@types/react-router-dom": "^5.1.6",
"@types/react-syntax-highlighter": "15.5.7",
"@types/react-syntax-highlighter": "15.5.13",
"@types/redux-mock-store": "1.0.4",
"@types/styled-components": "^5.1.4",
"@types/uuid": "^8.3.1",

View File

@@ -0,0 +1,13 @@
#!/usr/bin/env bash
# Extracts unique fenced code block language identifiers from all .md files under frontend/src/
# Usage: bash frontend/scripts/extract-md-languages.sh
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SRC_DIR="$SCRIPT_DIR/../src"
grep -roh '```[a-zA-Z0-9_+-]*' "$SRC_DIR" --include='*.md' \
| sed 's/^```//' \
| grep -v '^$' \
| sort -u

View File

@@ -0,0 +1,35 @@
#!/usr/bin/env bash
# Validates that all fenced code block languages used in .md files are registered
# in the syntax highlighter.
# Usage: bash frontend/scripts/validate-md-languages.sh
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SYNTAX_HIGHLIGHTER="$SCRIPT_DIR/../src/components/MarkdownRenderer/syntaxHighlighter.ts"
# Get all languages used in .md files
md_languages=$("$SCRIPT_DIR/extract-md-languages.sh")
# Get all registered languages from syntaxHighlighter.ts
registered_languages=$(grep -oP "registerLanguage\('\K[^']+" "$SYNTAX_HIGHLIGHTER" | sort -u)
missing_languages=()
for lang in $md_languages; do
if ! echo "$registered_languages" | grep -qx "$lang"; then
missing_languages+=("$lang")
fi
done
if [ ${#missing_languages[@]} -gt 0 ]; then
echo "Error: The following languages are used in .md files but not registered in syntaxHighlighter.ts:"
for lang in "${missing_languages[@]}"; do
echo " - $lang"
done
echo ""
echo "Please add them to: frontend/src/components/MarkdownRenderer/syntaxHighlighter.ts"
exit 1
fi
echo "All markdown code block languages are registered in syntaxHighlighter.ts"

View File

@@ -2,13 +2,12 @@
import ReactMarkdown from 'react-markdown';
import { CodeProps } from 'react-markdown/lib/ast-to-react';
import { Prism as SyntaxHighlighter } from 'react-syntax-highlighter';
import { a11yDark } from 'react-syntax-highlighter/dist/cjs/styles/prism';
import logEvent from 'api/common/logEvent';
import { isEmpty } from 'lodash-es';
import rehypeRaw from 'rehype-raw';
import CodeCopyBtn from './CodeCopyBtn/CodeCopyBtn';
import SyntaxHighlighter, { a11yDark } from './syntaxHighlighter';
interface LinkProps {
href: string;

View File

@@ -0,0 +1,34 @@
import { PrismLight as SyntaxHighlighter } from 'react-syntax-highlighter';
import bash from 'react-syntax-highlighter/dist/esm/languages/prism/bash';
import docker from 'react-syntax-highlighter/dist/esm/languages/prism/docker';
import elixir from 'react-syntax-highlighter/dist/esm/languages/prism/elixir';
import go from 'react-syntax-highlighter/dist/esm/languages/prism/go';
import javascript from 'react-syntax-highlighter/dist/esm/languages/prism/javascript';
import json from 'react-syntax-highlighter/dist/esm/languages/prism/json';
import jsx from 'react-syntax-highlighter/dist/esm/languages/prism/jsx';
import rust from 'react-syntax-highlighter/dist/esm/languages/prism/rust';
import swift from 'react-syntax-highlighter/dist/esm/languages/prism/swift';
import tsx from 'react-syntax-highlighter/dist/esm/languages/prism/tsx';
import typescript from 'react-syntax-highlighter/dist/esm/languages/prism/typescript';
import yaml from 'react-syntax-highlighter/dist/esm/languages/prism/yaml';
import a11yDark from 'react-syntax-highlighter/dist/esm/styles/prism/a11y-dark';
SyntaxHighlighter.registerLanguage('bash', bash);
SyntaxHighlighter.registerLanguage('docker', docker);
SyntaxHighlighter.registerLanguage('dockerfile', docker);
SyntaxHighlighter.registerLanguage('elixir', elixir);
SyntaxHighlighter.registerLanguage('go', go);
SyntaxHighlighter.registerLanguage('javascript', javascript);
SyntaxHighlighter.registerLanguage('js', javascript);
SyntaxHighlighter.registerLanguage('json', json);
SyntaxHighlighter.registerLanguage('jsx', jsx);
SyntaxHighlighter.registerLanguage('rust', rust);
SyntaxHighlighter.registerLanguage('swift', swift);
SyntaxHighlighter.registerLanguage('ts', typescript);
SyntaxHighlighter.registerLanguage('tsx', tsx);
SyntaxHighlighter.registerLanguage('typescript', typescript);
SyntaxHighlighter.registerLanguage('yaml', yaml);
SyntaxHighlighter.registerLanguage('yml', yaml);
export default SyntaxHighlighter;
export { a11yDark };

View File

@@ -6205,10 +6205,10 @@
"@types/history" "^4.7.11"
"@types/react" "*"
"@types/react-syntax-highlighter@15.5.7":
version "15.5.7"
resolved "https://registry.yarnpkg.com/@types/react-syntax-highlighter/-/react-syntax-highlighter-15.5.7.tgz#bd29020ccb118543d88779848f99059b64b02d0f"
integrity sha512-bo5fEO5toQeyCp0zVHBeggclqf5SQ/Z5blfFmjwO5dkMVGPgmiwZsJh9nu/Bo5L7IHTuGWrja6LxJVE2uB5ZrQ==
"@types/react-syntax-highlighter@15.5.13":
version "15.5.13"
resolved "https://registry.yarnpkg.com/@types/react-syntax-highlighter/-/react-syntax-highlighter-15.5.13.tgz#c5baf62a3219b3bf28d39cfea55d0a49a263d1f2"
integrity sha512-uLGJ87j6Sz8UaBAooU0T6lWJ0dBmjZgN1PZTrj05TNql2/XpC6+4HhMT5syIdFUUt+FASfCeLLv4kBygNU+8qA==
dependencies:
"@types/react" "*"

View File

@@ -1,62 +0,0 @@
package cloudintegration
import (
"context"
"net/http"
"github.com/SigNoz/signoz/pkg/types/cloudintegrationtypes"
"github.com/SigNoz/signoz/pkg/types/dashboardtypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
type Module interface {
GetName() cloudintegrationtypes.CloudProviderType
// AgentCheckIn is called by agent to heartbeat and get latest config in response.
AgentCheckIn(ctx context.Context, req *cloudintegrationtypes.PostableAgentCheckInPayload) (any, error)
GenerateConnectionParams(ctx context.Context) (*cloudintegrationtypes.GettableCloudIntegrationConnectionParams, error)
// GenerateConnectionArtifact generates cloud provider specific connection information, client side handles how this information is shown
GenerateConnectionArtifact(ctx context.Context, req *cloudintegrationtypes.PostableConnectionArtifact) (any, error)
// GetAccountStatus returns agent connection status for a cloud integration account
GetAccountStatus(ctx context.Context, orgID, accountID string) (*cloudintegrationtypes.GettableAccountStatus, error)
// ListConnectedAccounts lists accounts where agent is connected
ListConnectedAccounts(ctx context.Context, orgID string) (*cloudintegrationtypes.GettableConnectedAccountsList, error)
// LIstServices return list of services for a cloud provider attached with the accountID. This just returns a summary
ListServices(ctx context.Context, orgID string, accountID *string) (any, error) // returns either GettableAWSServices or GettableAzureServices
// GetServiceDetails returns service definition details for a serviceId. This returns config and other details required to show in service details page on client.
GetServiceDetails(ctx context.Context, req *cloudintegrationtypes.GetServiceDetailsReq) (any, error)
// GetDashboard returns dashboard json for a give cloud integration service dashboard.
// this only returns the dashboard when account is connected and service is enabled
GetDashboard(ctx context.Context, id string, orgID valuer.UUID) (*dashboardtypes.Dashboard, error)
// GetAvailableDashboards returns list of available dashboards across all connected cloud integration accounts in the org.
// this list gets added to dashboard list page
GetAvailableDashboards(ctx context.Context, orgID valuer.UUID) ([]*dashboardtypes.Dashboard, error)
// UpdateAccountConfig updates cloud integration account config
UpdateAccountConfig(ctx context.Context, orgId valuer.UUID, accountId string, config []byte) (any, error)
// UpdateServiceConfig updates cloud integration service config
UpdateServiceConfig(ctx context.Context, serviceId string, orgID valuer.UUID, config []byte) (any, error)
// DisconnectAccount soft deletes/removes a cloud integration account.
DisconnectAccount(ctx context.Context, orgID, accountID string) (*cloudintegrationtypes.CloudIntegration, error)
}
type Handler interface {
AgentCheckIn(http.ResponseWriter, *http.Request)
GenerateConnectionParams(http.ResponseWriter, *http.Request)
GenerateConnectionArtifact(http.ResponseWriter, *http.Request)
ListConnectedAccounts(http.ResponseWriter, *http.Request)
GetAccountStatus(http.ResponseWriter, *http.Request)
ListServices(http.ResponseWriter, *http.Request)
GetServiceDetails(http.ResponseWriter, *http.Request)
UpdateAccountConfig(http.ResponseWriter, *http.Request)
UpdateServiceConfig(http.ResponseWriter, *http.Request)
DisconnectAccount(http.ResponseWriter, *http.Request)
}

View File

@@ -12,6 +12,7 @@ import (
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/savedviewtypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
@@ -24,7 +25,7 @@ func NewModule(sqlstore sqlstore.SQLStore) savedview.Module {
}
func (module *module) GetViewsForFilters(ctx context.Context, orgID string, sourcePage string, name string, category string) ([]*v3.SavedView, error) {
var views []types.SavedView
var views []savedviewtypes.SavedView
var err error
if len(category) == 0 {
err = module.sqlstore.BunDB().NewSelect().Model(&views).Where("org_id = ? AND source_page = ? AND name LIKE ?", orgID, sourcePage, "%"+name+"%").Scan(ctx)
@@ -76,7 +77,7 @@ func (module *module) CreateView(ctx context.Context, orgID string, view v3.Save
createBy := claims.Email
updatedBy := claims.Email
dbView := types.SavedView{
dbView := savedviewtypes.SavedView{
TimeAuditable: types.TimeAuditable{
CreatedAt: createdAt,
UpdatedAt: updatedAt,
@@ -105,7 +106,7 @@ func (module *module) CreateView(ctx context.Context, orgID string, view v3.Save
}
func (module *module) GetView(ctx context.Context, orgID string, uuid valuer.UUID) (*v3.SavedView, error) {
var view types.SavedView
var view savedviewtypes.SavedView
err := module.sqlstore.BunDB().NewSelect().Model(&view).Where("org_id = ? AND id = ?", orgID, uuid.StringValue()).Scan(ctx)
if err != nil {
return nil, errors.WrapInternalf(err, errors.CodeInternal, "error in getting saved view")
@@ -146,7 +147,7 @@ func (module *module) UpdateView(ctx context.Context, orgID string, uuid valuer.
updatedBy := claims.Email
_, err = module.sqlstore.BunDB().NewUpdate().
Model(&types.SavedView{}).
Model(&savedviewtypes.SavedView{}).
Set("updated_at = ?, updated_by = ?, name = ?, category = ?, source_page = ?, tags = ?, data = ?, extra_data = ?",
updatedAt, updatedBy, view.Name, view.Category, view.SourcePage, strings.Join(view.Tags, ","), data, view.ExtraData).
Where("id = ?", uuid.StringValue()).
@@ -160,7 +161,7 @@ func (module *module) UpdateView(ctx context.Context, orgID string, uuid valuer.
func (module *module) DeleteView(ctx context.Context, orgID string, uuid valuer.UUID) error {
_, err := module.sqlstore.BunDB().NewDelete().
Model(&types.SavedView{}).
Model(&savedviewtypes.SavedView{}).
Where("id = ?", uuid.StringValue()).
Where("org_id = ?", orgID).
Exec(ctx)
@@ -171,7 +172,7 @@ func (module *module) DeleteView(ctx context.Context, orgID string, uuid valuer.
}
func (module *module) Collect(ctx context.Context, orgID valuer.UUID) (map[string]any, error) {
savedViews := []*types.SavedView{}
savedViews := []*savedviewtypes.SavedView{}
err := module.
sqlstore.
@@ -184,5 +185,5 @@ func (module *module) Collect(ctx context.Context, orgID valuer.UUID) (map[strin
return nil, err
}
return types.NewStatsFromSavedViews(savedViews), nil
return savedviewtypes.NewStatsFromSavedViews(savedViews), nil
}

View File

@@ -13,6 +13,7 @@ import (
root "github.com/SigNoz/signoz/pkg/modules/user"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/integrationtypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/gorilla/mux"
)
@@ -462,7 +463,7 @@ func (h *handler) UpdateAPIKey(w http.ResponseWriter, r *http.Request) {
return
}
if slices.Contains(types.AllIntegrationUserEmails, types.IntegrationUserEmail(createdByUser.Email.String())) {
if slices.Contains(integrationtypes.AllIntegrationUserEmails, integrationtypes.IntegrationUserEmail(createdByUser.Email.String())) {
render.Error(w, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "API Keys for integration users cannot be revoked"))
return
}
@@ -507,7 +508,7 @@ func (h *handler) RevokeAPIKey(w http.ResponseWriter, r *http.Request) {
return
}
if slices.Contains(types.AllIntegrationUserEmails, types.IntegrationUserEmail(createdByUser.Email.String())) {
if slices.Contains(integrationtypes.AllIntegrationUserEmails, integrationtypes.IntegrationUserEmail(createdByUser.Email.String())) {
render.Error(w, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "API Keys for integration users cannot be revoked"))
return
}

View File

@@ -19,6 +19,7 @@ import (
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/emailtypes"
"github.com/SigNoz/signoz/pkg/types/integrationtypes"
"github.com/SigNoz/signoz/pkg/types/roletypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/dustin/go-humanize"
@@ -279,7 +280,7 @@ func (module *Module) DeleteUser(ctx context.Context, orgID valuer.UUID, id stri
return errors.WithAdditionalf(err, "cannot delete root user")
}
if slices.Contains(types.AllIntegrationUserEmails, types.IntegrationUserEmail(user.Email.String())) {
if slices.Contains(integrationtypes.AllIntegrationUserEmails, integrationtypes.IntegrationUserEmail(user.Email.String())) {
return errors.New(errors.TypeForbidden, errors.CodeForbidden, "integration user cannot be deleted")
}

View File

@@ -10,15 +10,16 @@ import (
"github.com/SigNoz/signoz/pkg/query-service/model"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/integrationtypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
type cloudProviderAccountsRepository interface {
listConnected(ctx context.Context, orgId string, provider string) ([]types.CloudIntegration, *model.ApiError)
listConnected(ctx context.Context, orgId string, provider string) ([]integrationtypes.CloudIntegration, *model.ApiError)
get(ctx context.Context, orgId string, provider string, id string) (*types.CloudIntegration, *model.ApiError)
get(ctx context.Context, orgId string, provider string, id string) (*integrationtypes.CloudIntegration, *model.ApiError)
getConnectedCloudAccount(ctx context.Context, orgId string, provider string, accountID string) (*types.CloudIntegration, *model.ApiError)
getConnectedCloudAccount(ctx context.Context, orgId string, provider string, accountID string) (*integrationtypes.CloudIntegration, *model.ApiError)
// Insert an account or update it by (cloudProvider, id)
// for specified non-empty fields
@@ -27,11 +28,11 @@ type cloudProviderAccountsRepository interface {
orgId string,
provider string,
id *string,
config *types.AccountConfig,
config *integrationtypes.AccountConfig,
accountId *string,
agentReport *types.AgentReport,
agentReport *integrationtypes.AgentReport,
removedAt *time.Time,
) (*types.CloudIntegration, *model.ApiError)
) (*integrationtypes.CloudIntegration, *model.ApiError)
}
func newCloudProviderAccountsRepository(store sqlstore.SQLStore) (
@@ -48,8 +49,8 @@ type cloudProviderAccountsSQLRepository struct {
func (r *cloudProviderAccountsSQLRepository) listConnected(
ctx context.Context, orgId string, cloudProvider string,
) ([]types.CloudIntegration, *model.ApiError) {
accounts := []types.CloudIntegration{}
) ([]integrationtypes.CloudIntegration, *model.ApiError) {
accounts := []integrationtypes.CloudIntegration{}
err := r.store.BunDB().NewSelect().
Model(&accounts).
@@ -72,8 +73,8 @@ func (r *cloudProviderAccountsSQLRepository) listConnected(
func (r *cloudProviderAccountsSQLRepository) get(
ctx context.Context, orgId string, provider string, id string,
) (*types.CloudIntegration, *model.ApiError) {
var result types.CloudIntegration
) (*integrationtypes.CloudIntegration, *model.ApiError) {
var result integrationtypes.CloudIntegration
err := r.store.BunDB().NewSelect().
Model(&result).
@@ -97,8 +98,8 @@ func (r *cloudProviderAccountsSQLRepository) get(
func (r *cloudProviderAccountsSQLRepository) getConnectedCloudAccount(
ctx context.Context, orgId string, provider string, accountId string,
) (*types.CloudIntegration, *model.ApiError) {
var result types.CloudIntegration
) (*integrationtypes.CloudIntegration, *model.ApiError) {
var result integrationtypes.CloudIntegration
err := r.store.BunDB().NewSelect().
Model(&result).
@@ -127,11 +128,11 @@ func (r *cloudProviderAccountsSQLRepository) upsert(
orgId string,
provider string,
id *string,
config *types.AccountConfig,
config *integrationtypes.AccountConfig,
accountId *string,
agentReport *types.AgentReport,
agentReport *integrationtypes.AgentReport,
removedAt *time.Time,
) (*types.CloudIntegration, *model.ApiError) {
) (*integrationtypes.CloudIntegration, *model.ApiError) {
// Insert
if id == nil {
temp := valuer.GenerateUUID().StringValue()
@@ -181,7 +182,7 @@ func (r *cloudProviderAccountsSQLRepository) upsert(
)
}
integration := types.CloudIntegration{
integration := integrationtypes.CloudIntegration{
OrgID: orgId,
Provider: provider,
Identifiable: types.Identifiable{ID: valuer.MustNewUUID(*id)},

View File

@@ -14,6 +14,7 @@ import (
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/dashboardtypes"
"github.com/SigNoz/signoz/pkg/types/integrationtypes"
"github.com/SigNoz/signoz/pkg/valuer"
"golang.org/x/exp/maps"
)
@@ -52,7 +53,7 @@ func NewController(sqlStore sqlstore.SQLStore) (*Controller, error) {
}
type ConnectedAccountsListResponse struct {
Accounts []types.Account `json:"accounts"`
Accounts []integrationtypes.Account `json:"accounts"`
}
func (c *Controller) ListConnectedAccounts(ctx context.Context, orgId string, cloudProvider string) (
@@ -67,7 +68,7 @@ func (c *Controller) ListConnectedAccounts(ctx context.Context, orgId string, cl
return nil, model.WrapApiError(apiErr, "couldn't list cloud accounts")
}
connectedAccounts := []types.Account{}
connectedAccounts := []integrationtypes.Account{}
for _, a := range accountRecords {
connectedAccounts = append(connectedAccounts, a.Account())
}
@@ -81,7 +82,7 @@ type GenerateConnectionUrlRequest struct {
// Optional. To be specified for updates.
AccountId *string `json:"account_id,omitempty"`
AccountConfig types.AccountConfig `json:"account_config"`
AccountConfig integrationtypes.AccountConfig `json:"account_config"`
AgentConfig SigNozAgentConfig `json:"agent_config"`
}
@@ -149,9 +150,9 @@ func (c *Controller) GenerateConnectionUrl(ctx context.Context, orgId string, cl
}
type AccountStatusResponse struct {
Id string `json:"id"`
CloudAccountId *string `json:"cloud_account_id,omitempty"`
Status types.AccountStatus `json:"status"`
Id string `json:"id"`
CloudAccountId *string `json:"cloud_account_id,omitempty"`
Status integrationtypes.AccountStatus `json:"status"`
}
func (c *Controller) GetAccountStatus(ctx context.Context, orgId string, cloudProvider string, accountId string) (
@@ -217,7 +218,7 @@ func (c *Controller) CheckInAsAgent(ctx context.Context, orgId string, cloudProv
))
}
agentReport := types.AgentReport{
agentReport := integrationtypes.AgentReport{
TimestampMillis: time.Now().UnixMilli(),
Data: req.Data,
}
@@ -286,10 +287,10 @@ func (c *Controller) CheckInAsAgent(ctx context.Context, orgId string, cloudProv
}
type UpdateAccountConfigRequest struct {
Config types.AccountConfig `json:"config"`
Config integrationtypes.AccountConfig `json:"config"`
}
func (c *Controller) UpdateAccountConfig(ctx context.Context, orgId string, cloudProvider string, accountId string, req UpdateAccountConfigRequest) (*types.Account, *model.ApiError) {
func (c *Controller) UpdateAccountConfig(ctx context.Context, orgId string, cloudProvider string, accountId string, req UpdateAccountConfigRequest) (*integrationtypes.Account, *model.ApiError) {
if apiErr := validateCloudProviderName(cloudProvider); apiErr != nil {
return nil, apiErr
}
@@ -306,7 +307,7 @@ func (c *Controller) UpdateAccountConfig(ctx context.Context, orgId string, clou
return &account, nil
}
func (c *Controller) DisconnectAccount(ctx context.Context, orgId string, cloudProvider string, accountId string) (*types.CloudIntegration, *model.ApiError) {
func (c *Controller) DisconnectAccount(ctx context.Context, orgId string, cloudProvider string, accountId string) (*integrationtypes.CloudIntegration, *model.ApiError) {
if apiErr := validateCloudProviderName(cloudProvider); apiErr != nil {
return nil, apiErr
}
@@ -346,7 +347,7 @@ func (c *Controller) ListServices(
return nil, model.WrapApiError(apiErr, "couldn't list cloud services")
}
svcConfigs := map[string]*types.CloudServiceConfig{}
svcConfigs := map[string]*integrationtypes.CloudServiceConfig{}
if cloudAccountId != nil {
activeAccount, apiErr := c.accountsRepo.getConnectedCloudAccount(
ctx, orgID, cloudProvider, *cloudAccountId,
@@ -441,8 +442,8 @@ func (c *Controller) GetServiceDetails(
}
type UpdateServiceConfigRequest struct {
CloudAccountId string `json:"cloud_account_id"`
Config types.CloudServiceConfig `json:"config"`
CloudAccountId string `json:"cloud_account_id"`
Config integrationtypes.CloudServiceConfig `json:"config"`
}
func (u *UpdateServiceConfigRequest) Validate(def *services.Definition) error {
@@ -460,8 +461,8 @@ func (u *UpdateServiceConfigRequest) Validate(def *services.Definition) error {
}
type UpdateServiceConfigResponse struct {
Id string `json:"id"`
Config types.CloudServiceConfig `json:"config"`
Id string `json:"id"`
Config integrationtypes.CloudServiceConfig `json:"config"`
}
func (c *Controller) UpdateServiceConfig(

View File

@@ -3,20 +3,20 @@ package cloudintegrations
import (
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/query-service/app/cloudintegrations/services"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/integrationtypes"
)
type ServiceSummary struct {
services.Metadata
Config *types.CloudServiceConfig `json:"config"`
Config *integrationtypes.CloudServiceConfig `json:"config"`
}
type ServiceDetails struct {
services.Definition
Config *types.CloudServiceConfig `json:"config"`
ConnectionStatus *ServiceConnectionStatus `json:"status,omitempty"`
Config *integrationtypes.CloudServiceConfig `json:"config"`
ConnectionStatus *ServiceConnectionStatus `json:"status,omitempty"`
}
type AccountStatus struct {
@@ -61,7 +61,7 @@ func NewCompiledCollectionStrategy(provider string) (*CompiledCollectionStrategy
// Helper for accumulating strategies for enabled services.
func AddServiceStrategy(serviceType string, cs *CompiledCollectionStrategy,
definitionStrat *services.CollectionStrategy, config *types.CloudServiceConfig) error {
definitionStrat *services.CollectionStrategy, config *integrationtypes.CloudServiceConfig) error {
if definitionStrat.Provider != cs.Provider {
return errors.NewInternalf(CodeMismatchCloudProvider, "can't add %s service strategy to compiled strategy for %s",
definitionStrat.Provider, cs.Provider)

View File

@@ -9,6 +9,7 @@ import (
"github.com/SigNoz/signoz/pkg/query-service/model"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/integrationtypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
@@ -18,7 +19,7 @@ type ServiceConfigDatabase interface {
orgID string,
cloudAccountId string,
serviceType string,
) (*types.CloudServiceConfig, *model.ApiError)
) (*integrationtypes.CloudServiceConfig, *model.ApiError)
upsert(
ctx context.Context,
@@ -26,15 +27,15 @@ type ServiceConfigDatabase interface {
cloudProvider string,
cloudAccountId string,
serviceId string,
config types.CloudServiceConfig,
) (*types.CloudServiceConfig, *model.ApiError)
config integrationtypes.CloudServiceConfig,
) (*integrationtypes.CloudServiceConfig, *model.ApiError)
getAllForAccount(
ctx context.Context,
orgID string,
cloudAccountId string,
) (
configsBySvcId map[string]*types.CloudServiceConfig,
configsBySvcId map[string]*integrationtypes.CloudServiceConfig,
apiErr *model.ApiError,
)
}
@@ -56,9 +57,9 @@ func (r *serviceConfigSQLRepository) get(
orgID string,
cloudAccountId string,
serviceType string,
) (*types.CloudServiceConfig, *model.ApiError) {
) (*integrationtypes.CloudServiceConfig, *model.ApiError) {
var result types.CloudIntegrationService
var result integrationtypes.CloudIntegrationService
err := r.store.BunDB().NewSelect().
Model(&result).
@@ -89,14 +90,14 @@ func (r *serviceConfigSQLRepository) upsert(
cloudProvider string,
cloudAccountId string,
serviceId string,
config types.CloudServiceConfig,
) (*types.CloudServiceConfig, *model.ApiError) {
config integrationtypes.CloudServiceConfig,
) (*integrationtypes.CloudServiceConfig, *model.ApiError) {
// get cloud integration id from account id
// if the account is not connected, we don't need to upsert the config
var cloudIntegrationId string
err := r.store.BunDB().NewSelect().
Model((*types.CloudIntegration)(nil)).
Model((*integrationtypes.CloudIntegration)(nil)).
Column("id").
Where("provider = ?", cloudProvider).
Where("account_id = ?", cloudAccountId).
@@ -111,7 +112,7 @@ func (r *serviceConfigSQLRepository) upsert(
))
}
serviceConfig := types.CloudIntegrationService{
serviceConfig := integrationtypes.CloudIntegrationService{
Identifiable: types.Identifiable{ID: valuer.GenerateUUID()},
TimeAuditable: types.TimeAuditable{
CreatedAt: time.Now(),
@@ -139,8 +140,8 @@ func (r *serviceConfigSQLRepository) getAllForAccount(
ctx context.Context,
orgID string,
cloudAccountId string,
) (map[string]*types.CloudServiceConfig, *model.ApiError) {
serviceConfigs := []types.CloudIntegrationService{}
) (map[string]*integrationtypes.CloudServiceConfig, *model.ApiError) {
serviceConfigs := []integrationtypes.CloudIntegrationService{}
err := r.store.BunDB().NewSelect().
Model(&serviceConfigs).
@@ -154,7 +155,7 @@ func (r *serviceConfigSQLRepository) getAllForAccount(
))
}
result := map[string]*types.CloudServiceConfig{}
result := map[string]*integrationtypes.CloudServiceConfig{}
for _, r := range serviceConfigs {
result[r.Type] = &r.Config

View File

@@ -11,6 +11,7 @@ import (
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/dashboardtypes"
"github.com/SigNoz/signoz/pkg/types/integrationtypes"
"github.com/SigNoz/signoz/pkg/types/pipelinetypes"
ruletypes "github.com/SigNoz/signoz/pkg/types/ruletypes"
"github.com/SigNoz/signoz/pkg/valuer"
@@ -107,7 +108,7 @@ type IntegrationsListItem struct {
type Integration struct {
IntegrationDetails
Installation *types.InstalledIntegration `json:"installation"`
Installation *integrationtypes.InstalledIntegration `json:"installation"`
}
type Manager struct {
@@ -223,7 +224,7 @@ func (m *Manager) InstallIntegration(
ctx context.Context,
orgId string,
integrationId string,
config types.InstalledIntegrationConfig,
config integrationtypes.InstalledIntegrationConfig,
) (*IntegrationsListItem, *model.ApiError) {
integrationDetails, apiErr := m.getIntegrationDetails(ctx, integrationId)
if apiErr != nil {
@@ -429,7 +430,7 @@ func (m *Manager) getInstalledIntegration(
ctx context.Context,
orgId string,
integrationId string,
) (*types.InstalledIntegration, *model.ApiError) {
) (*integrationtypes.InstalledIntegration, *model.ApiError) {
iis, apiErr := m.installedIntegrationsRepo.get(
ctx, orgId, []string{integrationId},
)
@@ -457,7 +458,7 @@ func (m *Manager) getInstalledIntegrations(
return nil, apiErr
}
installedTypes := utils.MapSlice(installations, func(i types.InstalledIntegration) string {
installedTypes := utils.MapSlice(installations, func(i integrationtypes.InstalledIntegration) string {
return i.Type
})
integrationDetails, apiErr := m.availableIntegrationsRepo.get(ctx, installedTypes)

View File

@@ -4,22 +4,22 @@ import (
"context"
"github.com/SigNoz/signoz/pkg/query-service/model"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/integrationtypes"
)
type InstalledIntegrationsRepo interface {
list(ctx context.Context, orgId string) ([]types.InstalledIntegration, *model.ApiError)
list(ctx context.Context, orgId string) ([]integrationtypes.InstalledIntegration, *model.ApiError)
get(
ctx context.Context, orgId string, integrationTypes []string,
) (map[string]types.InstalledIntegration, *model.ApiError)
) (map[string]integrationtypes.InstalledIntegration, *model.ApiError)
upsert(
ctx context.Context,
orgId string,
integrationType string,
config types.InstalledIntegrationConfig,
) (*types.InstalledIntegration, *model.ApiError)
config integrationtypes.InstalledIntegrationConfig,
) (*integrationtypes.InstalledIntegration, *model.ApiError)
delete(ctx context.Context, orgId string, integrationType string) *model.ApiError
}

View File

@@ -7,6 +7,7 @@ import (
"github.com/SigNoz/signoz/pkg/query-service/model"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/integrationtypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/uptrace/bun"
)
@@ -26,8 +27,8 @@ func NewInstalledIntegrationsSqliteRepo(store sqlstore.SQLStore) (
func (r *InstalledIntegrationsSqliteRepo) list(
ctx context.Context,
orgId string,
) ([]types.InstalledIntegration, *model.ApiError) {
integrations := []types.InstalledIntegration{}
) ([]integrationtypes.InstalledIntegration, *model.ApiError) {
integrations := []integrationtypes.InstalledIntegration{}
err := r.store.BunDB().NewSelect().
Model(&integrations).
@@ -44,8 +45,8 @@ func (r *InstalledIntegrationsSqliteRepo) list(
func (r *InstalledIntegrationsSqliteRepo) get(
ctx context.Context, orgId string, integrationTypes []string,
) (map[string]types.InstalledIntegration, *model.ApiError) {
integrations := []types.InstalledIntegration{}
) (map[string]integrationtypes.InstalledIntegration, *model.ApiError) {
integrations := []integrationtypes.InstalledIntegration{}
typeValues := []interface{}{}
for _, integrationType := range integrationTypes {
@@ -62,7 +63,7 @@ func (r *InstalledIntegrationsSqliteRepo) get(
))
}
result := map[string]types.InstalledIntegration{}
result := map[string]integrationtypes.InstalledIntegration{}
for _, ii := range integrations {
result[ii.Type] = ii
}
@@ -74,10 +75,10 @@ func (r *InstalledIntegrationsSqliteRepo) upsert(
ctx context.Context,
orgId string,
integrationType string,
config types.InstalledIntegrationConfig,
) (*types.InstalledIntegration, *model.ApiError) {
config integrationtypes.InstalledIntegrationConfig,
) (*integrationtypes.InstalledIntegration, *model.ApiError) {
integration := types.InstalledIntegration{
integration := integrationtypes.InstalledIntegration{
Identifiable: types.Identifiable{
ID: valuer.GenerateUUID(),
},
@@ -114,7 +115,7 @@ func (r *InstalledIntegrationsSqliteRepo) delete(
ctx context.Context, orgId string, integrationType string,
) *model.ApiError {
_, dbErr := r.store.BunDB().NewDelete().
Model(&types.InstalledIntegration{}).
Model(&integrationtypes.InstalledIntegration{}).
Where("type = ?", integrationType).
Where("org_id = ?", orgId).
Exec(ctx)

View File

@@ -1,643 +0,0 @@
// NOTE:
// - When Account keyword is used in struct names, it refers cloud integration account. CloudIntegration refers to DB schema.
// - When Account Config keyword is used in struct names, it refers to configuration for cloud integration accounts
// - When Service keyword is used in struct names, it refers to cloud integration service. CloudIntegrationService refers to DB schema.
// where `service` is services provided by each cloud provider like AWS S3, Azure BlobStorage etc.
// - When Service Config keyword is used in struct names, it refers to configuration for cloud integration services
package cloudintegrationtypes
import (
"database/sql/driver"
"encoding/json"
"strings"
"time"
"github.com/uptrace/bun"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/valuer"
)
// CloudProviderType type alias
type CloudProviderType struct{ valuer.String }
var (
CloudProviderTypeAWS = valuer.NewString("aws")
CloudProviderTypeAzure = valuer.NewString("azure")
)
var ErrCodeCloudProviderInvalidInput = errors.MustNewCode("invalid_cloud_provider")
// NewCloudProvider returns a new CloudProviderType from a string. It validates the input and returns an error if the input is not valid.
func NewCloudProvider(provider string) (CloudProviderType, error) {
switch provider {
case CloudProviderTypeAWS.String():
return CloudProviderType{CloudProviderTypeAWS}, nil
case CloudProviderTypeAzure.String():
return CloudProviderType{CloudProviderTypeAzure}, nil
default:
return CloudProviderType{}, errors.NewInvalidInputf(ErrCodeCloudProviderInvalidInput, "invalid cloud provider: %s", provider)
}
}
var (
AWSIntegrationUserEmail = valuer.MustNewEmail("aws-integration@signoz.io")
AzureIntegrationUserEmail = valuer.MustNewEmail("azure-integration@signoz.io")
)
// CloudIntegrationUserEmails is the list of valid emails for Cloud One Click integrations.
// This is used for validation and restrictions in different contexts, across codebase.
var CloudIntegrationUserEmails = []valuer.Email{
AWSIntegrationUserEmail,
AzureIntegrationUserEmail,
}
func IsCloudIntegrationDashboardUuid(dashboardUuid string) bool {
parts := strings.SplitN(dashboardUuid, "--", 4)
if len(parts) != 4 {
return false
}
return parts[0] == "cloud-integration"
}
// GetCloudIntegrationDashboardID returns the cloud provider from dashboard id, if it's a cloud integration dashboard id.
// throws an error if invalid format or invalid cloud provider is provided in the dashboard id.
func GetCloudProviderFromDashboardID(dashboardUuid string) (CloudProviderType, error) {
parts := strings.SplitN(dashboardUuid, "--", 4)
if len(parts) != 4 {
return CloudProviderType{}, errors.NewInvalidInputf(ErrCodeCloudProviderInvalidInput, "invalid dashboard uuid: %s", dashboardUuid)
}
providerStr := parts[1]
cloudProvider, err := NewCloudProvider(providerStr)
if err != nil {
return CloudProviderType{}, err
}
return cloudProvider, nil
}
// Generic utility functions for JSON serialization/deserialization
// this is helpful to return right errors from a common place and avoid repeating the same code in multiple places.
// UnmarshalJSON is a generic function to unmarshal JSON data into any type
func UnmarshalJSON[T any](src []byte, target *T) error {
err := json.Unmarshal(src, target)
if err != nil {
return errors.WrapInternalf(
err, errors.CodeInternal, "couldn't deserialize JSON",
)
}
return nil
}
// MarshalJSON is a generic function to marshal any type to JSON
func MarshalJSON[T any](source *T) ([]byte, error) {
if source == nil {
return nil, errors.NewInternalf(errors.CodeInternal, "source is nil")
}
serialized, err := json.Marshal(source)
if err != nil {
return nil, errors.WrapInternalf(
err, errors.CodeInternal, "couldn't serialize to JSON",
)
}
return serialized, nil
}
// GettableConnectedAccountsList is the response for listing connected accounts for a cloud provider.
type GettableConnectedAccountsList struct {
Accounts []*Account `json:"accounts"`
}
// SigNozAgentConfig represents parameters required for agent deployment in cloud provider accounts
// these represent parameters passed during agent deployment, how they are passed might change for each cloud provider but the purpose is same.
type SigNozAgentConfig struct {
Region string `json:"region,omitempty"` // AWS-specific: The region in which SigNoz agent should be installed
IngestionUrl string `json:"ingestion_url"`
IngestionKey string `json:"ingestion_key"`
SigNozAPIUrl string `json:"signoz_api_url"`
SigNozAPIKey string `json:"signoz_api_key"`
Version string `json:"version,omitempty"`
}
// PostableConnectionArtifact represent request body for generating connection artifact API.
// Data is request body raw bytes since each cloud provider will have have different request body structure and generics hardly help in such cases.
// Artifact is a generic name for different types of connection methods like connection URL for AWS, connection command for Azure etc.
type PostableConnectionArtifact struct {
OrgID string
Data []byte // either PostableAWSConnectionUrl or PostableAzureConnectionCommand
}
// PostableAWSConnectionUrl is request body for AWS connection artifact API
type PostableAWSConnectionUrl struct {
AgentConfig *SigNozAgentConfig `json:"agent_config"`
AccountConfig *AWSAccountConfig `json:"account_config"`
}
// PostableAzureConnectionCommand is request body for Azure connection artifact API
type PostableAzureConnectionCommand struct {
AgentConfig *SigNozAgentConfig `json:"agent_config"`
AccountConfig *AzureAccountConfig `json:"account_config"`
}
// GettableAzureConnectionArtifact is Azure specific connection artifact which contains connection commands for agent deployment
type GettableAzureConnectionArtifact struct {
AzureShellConnectionCommand string `json:"az_shell_connection_command"`
AzureCliConnectionCommand string `json:"az_cli_connection_command"`
}
// GettableAWSConnectionUrl is AWS specific connection artifact which contains connection url for agent deployment
type GettableAWSConnectionUrl struct {
AccountId string `json:"account_id"`
ConnectionUrl string `json:"connection_url"`
}
// GettableAzureConnectionCommand is Azure specific connection artifact which contains connection commands for agent deployment
type GettableAzureConnectionCommand struct {
AccountId string `json:"account_id"`
AzureShellConnectionCommand string `json:"az_shell_connection_command"`
AzureCliConnectionCommand string `json:"az_cli_connection_command"`
}
// GettableAccountStatus is cloud integration account status response
type GettableAccountStatus struct {
Id string `json:"id"`
CloudAccountId *string `json:"cloud_account_id,omitempty"`
Status AccountStatus `json:"status"`
}
// PostableAgentCheckInPayload is request body for agent check-in API.
// This is used by agent to send heartbeat.
type PostableAgentCheckInPayload struct {
ID string `json:"account_id"`
AccountID string `json:"cloud_account_id"`
// Arbitrary cloud specific Agent data
Data map[string]any `json:"data,omitempty"`
OrgID string `json:"-"`
}
// AWSAgentIntegrationConfig is used by agent for deploying infra to send telemetry to SigNoz
type AWSAgentIntegrationConfig struct {
EnabledRegions []string `json:"enabled_regions"`
TelemetryCollectionStrategy *AWSCollectionStrategy `json:"telemetry,omitempty"`
}
// AzureAgentIntegrationConfig is used by agent for deploying infra to send telemetry to SigNoz
type AzureAgentIntegrationConfig struct {
DeploymentRegion string `json:"deployment_region"` // will not be changed once set
EnabledResourceGroups []string `json:"resource_groups"`
// TelemetryCollectionStrategy is map of service to telemetry config
TelemetryCollectionStrategy map[string]*AzureCollectionStrategy `json:"telemetry,omitempty"`
}
// GettableAgentCheckInRes is generic response from agent check-in API.
// AWSAgentIntegrationConfig and AzureAgentIntegrationConfig these configs are used by agent to deploy the infra and send telemetry to SigNoz
type GettableAgentCheckInRes[AgentConfigT any] struct {
AccountId string `json:"account_id"`
CloudAccountId string `json:"cloud_account_id"`
RemovedAt *time.Time `json:"removed_at"`
IntegrationConfig AgentConfigT `json:"integration_config"`
}
// UpdatableServiceConfig is generic
type UpdatableServiceConfig[ServiceConfigT any] struct {
CloudAccountId string `json:"cloud_account_id"`
Config ServiceConfigT `json:"config"`
}
// ServiceConfigTyped is a generic interface for cloud integration service's configuration
// this is generic interface to define helper functions for CloudIntegrationService.Config field.
type ServiceConfigTyped[definition Definition] interface {
Validate(def definition) error
IsMetricsEnabled() bool
IsLogsEnabled() bool
}
type AWSServiceConfig struct {
Logs *AWSServiceLogsConfig `json:"logs,omitempty"`
Metrics *AWSServiceMetricsConfig `json:"metrics,omitempty"`
}
type AWSServiceLogsConfig struct {
Enabled bool `json:"enabled"`
S3Buckets map[string][]string `json:"s3_buckets,omitempty"`
}
type AWSServiceMetricsConfig struct {
Enabled bool `json:"enabled"`
}
// IsMetricsEnabled returns true if metrics collection is configured and enabled
func (a *AWSServiceConfig) IsMetricsEnabled() bool {
return a.Metrics != nil && a.Metrics.Enabled
}
// IsLogsEnabled returns true if logs collection is configured and enabled
func (a *AWSServiceConfig) IsLogsEnabled() bool {
return a.Logs != nil && a.Logs.Enabled
}
type AzureServiceConfig struct {
Logs []*AzureServiceLogsConfig `json:"logs,omitempty"`
Metrics []*AzureServiceMetricsConfig `json:"metrics,omitempty"`
}
// AzureServiceLogsConfig is Azure specific service config for logs
type AzureServiceLogsConfig struct {
Enabled bool `json:"enabled"`
Name string `json:"name"`
}
// AzureServiceMetricsConfig is Azure specific service config for metrics
type AzureServiceMetricsConfig struct {
Enabled bool `json:"enabled"`
Name string `json:"name"`
}
// IsMetricsEnabled returns true if any metric is configured and enabled
func (a *AzureServiceConfig) IsMetricsEnabled() bool {
if a.Metrics == nil {
return false
}
for _, m := range a.Metrics {
if m.Enabled {
return true
}
}
return false
}
// IsLogsEnabled returns true if any log is configured and enabled
func (a *AzureServiceConfig) IsLogsEnabled() bool {
if a.Logs == nil {
return false
}
for _, l := range a.Logs {
if l.Enabled {
return true
}
}
return false
}
func (a *AWSServiceConfig) Validate(def *AWSDefinition) error {
if def.Id != S3Sync.String() && a.Logs != nil && a.Logs.S3Buckets != nil {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "s3 buckets can only be added to service-type[%s]", S3Sync)
} else if def.Id == S3Sync.String() && a.Logs != nil && a.Logs.S3Buckets != nil {
for region := range a.Logs.S3Buckets {
if _, found := ValidAWSRegions[region]; !found {
return errors.NewInvalidInputf(CodeInvalidCloudRegion, "invalid cloud region: %s", region)
}
}
}
return nil
}
func (a *AzureServiceConfig) Validate(def *AzureDefinition) error {
logsMap := make(map[string]bool)
metricsMap := make(map[string]bool)
if def.Strategy != nil && def.Strategy.Logs != nil {
for _, log := range def.Strategy.Logs {
logsMap[log.Name] = true
}
}
if def.Strategy != nil && def.Strategy.Metrics != nil {
for _, metric := range def.Strategy.Metrics {
metricsMap[metric.Name] = true
}
}
for _, log := range a.Logs {
if _, found := logsMap[log.Name]; !found {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "invalid log name: %s", log.Name)
}
}
for _, metric := range a.Metrics {
if _, found := metricsMap[metric.Name]; !found {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "invalid metric name: %s", metric.Name)
}
}
return nil
}
// UpdatableServiceConfigRes is response for UpdateServiceConfig API
// TODO: find a better way to name this
type UpdatableServiceConfigRes struct {
ServiceId string `json:"id"`
Config any `json:"config"`
}
// UpdatableAccountConfigTyped is a generic struct for updating cloud integration account config used in UpdateAccountConfig API
type UpdatableAccountConfigTyped[AccountConfigT any] struct {
Config *AccountConfigT `json:"config"`
}
type (
UpdatableAWSAccountConfig = UpdatableAccountConfigTyped[AWSAccountConfig]
UpdatableAzureAccountConfig = UpdatableAccountConfigTyped[AzureAccountConfig]
)
// AWSAccountConfig is the configuration for AWS cloud integration account
type AWSAccountConfig struct {
EnabledRegions []string `json:"regions"`
}
// AzureAccountConfig is the configuration for Azure cloud integration account
type AzureAccountConfig struct {
DeploymentRegion string `json:"deployment_region,omitempty"`
EnabledResourceGroups []string `json:"resource_groups,omitempty"`
}
// GettableServices is a generic struct for listing services of a cloud integration account used in ListServices API
type GettableServices[ServiceSummaryT any] struct {
Services []ServiceSummaryT `json:"services"`
}
type (
GettableAWSServices = GettableServices[AWSServiceSummary]
GettableAzureServices = GettableServices[AzureServiceSummary]
)
// GetServiceDetailsReq is a req struct for getting service definition details
type GetServiceDetailsReq struct {
OrgID valuer.UUID
ServiceId string
CloudAccountID *string
}
// ServiceSummary is a generic struct for service summary used in ListServices API
type ServiceSummary[ServiceConfigT any] struct {
DefinitionMetadata
Config *ServiceConfigT `json:"config"`
}
type (
AWSServiceSummary = ServiceSummary[AWSServiceConfig]
AzureServiceSummary = ServiceSummary[AzureServiceConfig]
)
// GettableServiceDetails is a generic struct for service details used in GetServiceDetails API
type GettableServiceDetails[DefinitionT any, ServiceConfigT any] struct {
Definition DefinitionT `json:",inline"`
Config ServiceConfigT `json:"config"`
ConnectionStatus *ServiceConnectionStatus `json:"status,omitempty"`
}
type (
GettableAWSServiceDetails = GettableServiceDetails[AWSDefinition, *AWSServiceConfig]
GettableAzureServiceDetails = GettableServiceDetails[AzureDefinition, *AzureServiceConfig]
)
// ServiceConnectionStatus represents integration connection status for a particular service
// this struct helps to check ingested data and determines connection status by whether data was ingested or not.
// this is composite struct for both metrics and logs
type ServiceConnectionStatus struct {
Logs []*SignalConnectionStatus `json:"logs"`
Metrics []*SignalConnectionStatus `json:"metrics"`
}
// SignalConnectionStatus represents connection status for a particular signal type (logs or metrics) for a service
// this struct is used in API responses for clients to show relevant information about the connection status.
type SignalConnectionStatus struct {
CategoryID string `json:"category"`
CategoryDisplayName string `json:"category_display_name"`
LastReceivedTsMillis int64 `json:"last_received_ts_ms"` // epoch milliseconds
LastReceivedFrom string `json:"last_received_from"` // resource identifier
}
// GettableCloudIntegrationConnectionParams is response for connection params API
type GettableCloudIntegrationConnectionParams struct {
IngestionUrl string `json:"ingestion_url,omitempty"`
IngestionKey string `json:"ingestion_key,omitempty"`
SigNozAPIUrl string `json:"signoz_api_url,omitempty"`
SigNozAPIKey string `json:"signoz_api_key,omitempty"`
}
// GettableIngestionKey is a struct for ingestion key returned from gateway
type GettableIngestionKey struct {
Name string `json:"name"`
Value string `json:"value"`
// other attributes from gateway response not included here since they are not being used.
}
// GettableIngestionKeysSearch is a struct for response of ingestion keys search API on gateway
type GettableIngestionKeysSearch struct {
Status string `json:"status"`
Data []GettableIngestionKey `json:"data"`
Error string `json:"error"`
}
// GettableCreateIngestionKey is a struct for response of create ingestion key API on gateway
type GettableCreateIngestionKey struct {
Status string `json:"status"`
Data GettableIngestionKey `json:"data"`
Error string `json:"error"`
}
// GettableDeployment is response struct for deployment details fetched from Zeus
type GettableDeployment struct {
Name string `json:"name"`
ClusterInfo struct {
Region struct {
DNS string `json:"dns"`
} `json:"region"`
} `json:"cluster"`
}
// --------------------------------------------------------------------------
// Cloud integration uses the cloud_integration table
// and cloud_integrations_service table
// --------------------------------------------------------------------------
type CloudIntegration struct {
bun.BaseModel `bun:"table:cloud_integration"`
types.Identifiable
types.TimeAuditable
Provider string `json:"provider" bun:"provider,type:text,unique:provider_id"`
Config *AccountConfig `json:"config" bun:"config,type:text"`
AccountID *string `json:"account_id" bun:"account_id,type:text"`
LastAgentReport *AgentReport `json:"last_agent_report" bun:"last_agent_report,type:text"`
RemovedAt *time.Time `json:"removed_at" bun:"removed_at,type:timestamp,nullzero"`
OrgID string `bun:"org_id,type:text,unique:provider_id"`
}
func (a *CloudIntegration) Status() AccountStatus {
status := AccountStatus{}
if a.LastAgentReport != nil {
lastHeartbeat := a.LastAgentReport.TimestampMillis
status.Integration.LastHeartbeatTsMillis = &lastHeartbeat
}
return status
}
func (a *CloudIntegration) Account() Account {
ca := Account{Id: a.ID.StringValue(), Status: a.Status()}
if a.AccountID != nil {
ca.CloudAccountId = *a.AccountID
}
if a.Config != nil {
ca.Config = *a.Config
} else {
ca.Config = DefaultAccountConfig()
}
return ca
}
type Account struct {
Id string `json:"id"`
CloudAccountId string `json:"cloud_account_id"`
Config AccountConfig `json:"config"`
Status AccountStatus `json:"status"`
}
type AccountStatus struct {
Integration AccountIntegrationStatus `json:"integration"`
}
type AccountIntegrationStatus struct {
LastHeartbeatTsMillis *int64 `json:"last_heartbeat_ts_ms"`
}
func DefaultAccountConfig() AccountConfig {
return AccountConfig{
EnabledRegions: []string{},
}
}
type AccountConfig struct {
EnabledRegions []string `json:"regions"`
}
// For serializing from db
func (c *AccountConfig) Scan(src any) error {
var data []byte
switch v := src.(type) {
case []byte:
data = v
case string:
data = []byte(v)
default:
return errors.NewInternalf(errors.CodeInternal, "tried to scan from %T instead of string or bytes", src)
}
return json.Unmarshal(data, c)
}
// For serializing to db
func (c *AccountConfig) Value() (driver.Value, error) {
if c == nil {
return nil, errors.NewInternalf(errors.CodeInternal, "cloud account config is nil")
}
serialized, err := json.Marshal(c)
if err != nil {
return nil, errors.WrapInternalf(err, errors.CodeInternal, "couldn't serialize cloud account config to JSON")
}
// Return as string instead of []byte to ensure PostgreSQL stores as text, not bytea
return string(serialized), nil
}
type AgentReport struct {
TimestampMillis int64 `json:"timestamp_millis"`
Data map[string]any `json:"data"`
}
// For serializing from db
func (r *AgentReport) Scan(src any) error {
var data []byte
switch v := src.(type) {
case []byte:
data = v
case string:
data = []byte(v)
default:
return errors.NewInternalf(errors.CodeInternal, "tried to scan from %T instead of string or bytes", src)
}
return json.Unmarshal(data, r)
}
// For serializing to db
func (r *AgentReport) Value() (driver.Value, error) {
if r == nil {
return nil, errors.NewInternalf(errors.CodeInternal, "agent report is nil")
}
serialized, err := json.Marshal(r)
if err != nil {
return nil, errors.WrapInternalf(
err, errors.CodeInternal, "couldn't serialize agent report to JSON",
)
}
// Return as string instead of []byte to ensure PostgreSQL stores as text, not bytea
return string(serialized), nil
}
type CloudIntegrationService struct {
bun.BaseModel `bun:"table:cloud_integration_service,alias:cis"`
types.Identifiable
types.TimeAuditable
Type string `bun:"type,type:text,notnull,unique:cloud_integration_id_type"`
Config CloudServiceConfig `bun:"config,type:text"`
CloudIntegrationID string `bun:"cloud_integration_id,type:text,notnull,unique:cloud_integration_id_type,references:cloud_integrations(id),on_delete:cascade"`
}
type CloudServiceLogsConfig struct {
Enabled bool `json:"enabled"`
S3Buckets map[string][]string `json:"s3_buckets,omitempty"`
}
type CloudServiceMetricsConfig struct {
Enabled bool `json:"enabled"`
}
type CloudServiceConfig struct {
Logs *CloudServiceLogsConfig `json:"logs,omitempty"`
Metrics *CloudServiceMetricsConfig `json:"metrics,omitempty"`
}
// For serializing from db
func (c *CloudServiceConfig) Scan(src any) error {
var data []byte
switch src := src.(type) {
case []byte:
data = src
case string:
data = []byte(src)
default:
return errors.NewInternalf(errors.CodeInternal, "tried to scan from %T instead of string or bytes", src)
}
return json.Unmarshal(data, c)
}
// For serializing to db
func (c *CloudServiceConfig) Value() (driver.Value, error) {
if c == nil {
return nil, errors.NewInternalf(errors.CodeInternal, "cloud service config is nil")
}
serialized, err := json.Marshal(c)
if err != nil {
return nil, errors.WrapInternalf(
err, errors.CodeInternal, "couldn't serialize cloud service config to JSON",
)
}
// Return as string instead of []byte to ensure PostgreSQL stores as text, not bytea
return string(serialized), nil
}

View File

@@ -1,103 +0,0 @@
package cloudintegrationtypes
import (
"github.com/SigNoz/signoz/pkg/errors"
)
var (
CodeInvalidCloudRegion = errors.MustNewCode("invalid_cloud_region")
CodeMismatchCloudProvider = errors.MustNewCode("cloud_provider_mismatch")
)
// List of all valid cloud regions on Amazon Web Services
var ValidAWSRegions = map[string]bool{
"af-south-1": true, // Africa (Cape Town).
"ap-east-1": true, // Asia Pacific (Hong Kong).
"ap-northeast-1": true, // Asia Pacific (Tokyo).
"ap-northeast-2": true, // Asia Pacific (Seoul).
"ap-northeast-3": true, // Asia Pacific (Osaka).
"ap-south-1": true, // Asia Pacific (Mumbai).
"ap-south-2": true, // Asia Pacific (Hyderabad).
"ap-southeast-1": true, // Asia Pacific (Singapore).
"ap-southeast-2": true, // Asia Pacific (Sydney).
"ap-southeast-3": true, // Asia Pacific (Jakarta).
"ap-southeast-4": true, // Asia Pacific (Melbourne).
"ca-central-1": true, // Canada (Central).
"ca-west-1": true, // Canada West (Calgary).
"eu-central-1": true, // Europe (Frankfurt).
"eu-central-2": true, // Europe (Zurich).
"eu-north-1": true, // Europe (Stockholm).
"eu-south-1": true, // Europe (Milan).
"eu-south-2": true, // Europe (Spain).
"eu-west-1": true, // Europe (Ireland).
"eu-west-2": true, // Europe (London).
"eu-west-3": true, // Europe (Paris).
"il-central-1": true, // Israel (Tel Aviv).
"me-central-1": true, // Middle East (UAE).
"me-south-1": true, // Middle East (Bahrain).
"sa-east-1": true, // South America (Sao Paulo).
"us-east-1": true, // US East (N. Virginia).
"us-east-2": true, // US East (Ohio).
"us-west-1": true, // US West (N. California).
"us-west-2": true, // US West (Oregon).
}
// List of all valid cloud regions for Microsoft Azure
var ValidAzureRegions = map[string]bool{
"australiacentral": true, // Australia Central
"australiacentral2": true, // Australia Central 2
"australiaeast": true, // Australia East
"australiasoutheast": true, // Australia Southeast
"austriaeast": true, // Austria East
"belgiumcentral": true, // Belgium Central
"brazilsouth": true, // Brazil South
"brazilsoutheast": true, // Brazil Southeast
"canadacentral": true, // Canada Central
"canadaeast": true, // Canada East
"centralindia": true, // Central India
"centralus": true, // Central US
"chilecentral": true, // Chile Central
"denmarkeast": true, // Denmark East
"eastasia": true, // East Asia
"eastus": true, // East US
"eastus2": true, // East US 2
"francecentral": true, // France Central
"francesouth": true, // France South
"germanynorth": true, // Germany North
"germanywestcentral": true, // Germany West Central
"indonesiacentral": true, // Indonesia Central
"israelcentral": true, // Israel Central
"italynorth": true, // Italy North
"japaneast": true, // Japan East
"japanwest": true, // Japan West
"koreacentral": true, // Korea Central
"koreasouth": true, // Korea South
"malaysiawest": true, // Malaysia West
"mexicocentral": true, // Mexico Central
"newzealandnorth": true, // New Zealand North
"northcentralus": true, // North Central US
"northeurope": true, // North Europe
"norwayeast": true, // Norway East
"norwaywest": true, // Norway West
"polandcentral": true, // Poland Central
"qatarcentral": true, // Qatar Central
"southafricanorth": true, // South Africa North
"southafricawest": true, // South Africa West
"southcentralus": true, // South Central US
"southindia": true, // South India
"southeastasia": true, // Southeast Asia
"spaincentral": true, // Spain Central
"swedencentral": true, // Sweden Central
"switzerlandnorth": true, // Switzerland North
"switzerlandwest": true, // Switzerland West
"uaecentral": true, // UAE Central
"uaenorth": true, // UAE North
"uksouth": true, // UK South
"ukwest": true, // UK West
"westcentralus": true, // West Central US
"westeurope": true, // West Europe
"westindia": true, // West India
"westus": true, // West US
"westus2": true, // West US 2
"westus3": true, // West US 3
}

View File

@@ -1,263 +0,0 @@
package cloudintegrationtypes
import (
"fmt"
"time"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/dashboardtypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
var S3Sync = valuer.NewString("s3sync")
// Generic interface for cloud service definition.
// This is implemented by AWSDefinition and AzureDefinition, which represent service definitions for AWS and Azure respectively.
// Generics work well so far because service definitions share a similar logic.
// We dont want to over-do generics as well, if the service definitions functionally diverge in the future consider breaking generics.
type Definition interface {
GetId() string
Validate() error
PopulateDashboardURLs(cloudProvider CloudProviderType, svcId string)
GetIngestionStatusCheck() *IngestionStatusCheck
GetAssets() Assets
}
// AWSDefinition represents AWS Service definition, which includes collection strategy, dashboards and meta info for integration
type AWSDefinition = ServiceDefinition[AWSCollectionStrategy]
// AzureDefinition represents Azure Service definition, which includes collection strategy, dashboards and meta info for integration
type AzureDefinition = ServiceDefinition[AzureCollectionStrategy]
// Making AWSDefinition and AzureDefinition satisfy Definition interface, so that they can be used in a generic way
var (
_ Definition = &AWSDefinition{}
_ Definition = &AzureDefinition{}
)
// ServiceDefinition represents generic struct for cloud service, regardless of the cloud provider.
// this struct must satify Definition interface.
// StrategyT is of either AWSCollectionStrategy or AzureCollectionStrategy, depending on the cloud provider.
type ServiceDefinition[StrategyT any] struct {
DefinitionMetadata
Overview string `json:"overview"` // markdown
Assets Assets `json:"assets"`
SupportedSignals SupportedSignals `json:"supported_signals"`
DataCollected DataCollected `json:"data_collected"`
IngestionStatusCheck *IngestionStatusCheck `json:"ingestion_status_check,omitempty"`
Strategy *StrategyT `json:"telemetry_collection_strategy"`
}
// Following methods are quite self explanatory, they are just to satisfy the Definition interface and provide some utility functions for service definitions.
func (def *ServiceDefinition[StrategyT]) GetId() string {
return def.Id
}
func (def *ServiceDefinition[StrategyT]) Validate() error {
seenDashboardIds := map[string]interface{}{}
if def.Strategy == nil {
return errors.NewInternalf(errors.CodeInternal, "telemetry_collection_strategy is required")
}
for _, dd := range def.Assets.Dashboards {
if _, seen := seenDashboardIds[dd.Id]; seen {
return errors.NewInternalf(errors.CodeInternal, "multiple dashboards found with id %s", dd.Id)
}
seenDashboardIds[dd.Id] = nil
}
return nil
}
func (def *ServiceDefinition[StrategyT]) PopulateDashboardURLs(cloudProvider CloudProviderType, svcId string) {
for i := range def.Assets.Dashboards {
dashboardId := def.Assets.Dashboards[i].Id
url := "/dashboard/" + GetCloudIntegrationDashboardID(cloudProvider, svcId, dashboardId)
def.Assets.Dashboards[i].Url = url
}
}
func (def *ServiceDefinition[StrategyT]) GetIngestionStatusCheck() *IngestionStatusCheck {
return def.IngestionStatusCheck
}
func (def *ServiceDefinition[StrategyT]) GetAssets() Assets {
return def.Assets
}
// DefinitionMetadata represents service definition metadata. This is useful for showing service overview
type DefinitionMetadata struct {
Id string `json:"id"`
Title string `json:"title"`
Icon string `json:"icon"`
}
// IngestionStatusCheckCategory represents a category of ingestion status check. Applies for both metrics and logs.
// A category can be "Overview" of metrics or "Enhanced" Metrics for AWS, and "Transaction" or "Capacity" metrics for Azure.
// Each category can have multiple checks (AND logic), if all checks pass,
// then we can be sure that data is being ingested for that category of the signal
type IngestionStatusCheckCategory struct {
Category string `json:"category"`
DisplayName string `json:"display_name"`
Checks []*IngestionStatusCheckAttribute `json:"checks"`
}
// IngestionStatusCheckAttribute represents a check or condition for ingestion status.
// Key can be metric name or part of log message
type IngestionStatusCheckAttribute struct {
Key string `json:"key"` // OPTIONAL search key (metric name or log message)
Attributes []*IngestionStatusCheckAttributeFilter `json:"attributes"`
}
// IngestionStatusCheck represents combined checks for metrics and logs for a service
type IngestionStatusCheck struct {
Metrics []*IngestionStatusCheckCategory `json:"metrics"`
Logs []*IngestionStatusCheckCategory `json:"logs"`
}
// IngestionStatusCheckAttributeFilter represents filter for a check, which can be used to filter specific log messages or metrics with specific attributes.
// For example, we can use it to filter logs with specific log level or metrics with specific dimensions.
type IngestionStatusCheckAttributeFilter struct {
Name string `json:"name"`
Operator string `json:"operator"`
Value string `json:"value"` // OPTIONAL
}
// Assets represents the collection of dashboards
type Assets struct {
Dashboards []Dashboard `json:"dashboards"`
}
// SupportedSignals for cloud provider's service
type SupportedSignals struct {
Logs bool `json:"logs"`
Metrics bool `json:"metrics"`
}
// DataCollected is curated static list of metrics and logs, this is shown as part of service overview
type DataCollected struct {
Logs []CollectedLogAttribute `json:"logs"`
Metrics []CollectedMetric `json:"metrics"`
}
// CollectedLogAttribute represents a log attribute that is present in all log entries for a service,
// this is shown as part of service overview
type CollectedLogAttribute struct {
Name string `json:"name"`
Path string `json:"path"`
Type string `json:"type"`
}
// CollectedMetric represents a metric that is collected for a service, this is shown as part of service overview
type CollectedMetric struct {
Name string `json:"name"`
Type string `json:"type"`
Unit string `json:"unit"`
Description string `json:"description"`
}
// AWSCollectionStrategy represents signal collection strategy for AWS services.
// this is AWS specific.
type AWSCollectionStrategy struct {
Metrics *AWSMetricsStrategy `json:"aws_metrics,omitempty"`
Logs *AWSLogsStrategy `json:"aws_logs,omitempty"`
S3Buckets map[string][]string `json:"s3_buckets,omitempty"` // Only available in S3 Sync Service Type in AWS
}
// AzureCollectionStrategy represents signal collection strategy for Azure services.
// this is Azure specific.
type AzureCollectionStrategy struct {
Metrics []*AzureMetricsStrategy `json:"azure_metrics,omitempty"`
Logs []*AzureLogsStrategy `json:"azure_logs,omitempty"`
}
// AWSMetricsStrategy represents metrics collection strategy for AWS services.
// this is AWS specific.
type AWSMetricsStrategy struct {
// to be used as https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-cloudwatch-metricstream.html#cfn-cloudwatch-metricstream-includefilters
StreamFilters []struct {
// json tags here are in the shape expected by AWS API as detailed at
// https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-cloudwatch-metricstream-metricstreamfilter.html
Namespace string `json:"Namespace"`
MetricNames []string `json:"MetricNames,omitempty"`
} `json:"cloudwatch_metric_stream_filters"`
}
// AWSLogsStrategy represents logs collection strategy for AWS services.
// this is AWS specific.
type AWSLogsStrategy struct {
Subscriptions []struct {
// subscribe to all logs groups with specified prefix.
// eg: `/aws/rds/`
LogGroupNamePrefix string `json:"log_group_name_prefix"`
// https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html
// "" implies no filtering is required.
FilterPattern string `json:"filter_pattern"`
} `json:"cloudwatch_logs_subscriptions"`
}
// AzureMetricsStrategy represents metrics collection strategy for Azure services.
// this is Azure specific.
type AzureMetricsStrategy struct {
CategoryType string `json:"category_type"`
Name string `json:"name"`
}
// AzureLogsStrategy represents logs collection strategy for Azure services.
// this is Azure specific. Even though this is similar to AzureMetricsStrategy, keeping it separate for future flexibility and clarity.
type AzureLogsStrategy struct {
CategoryType string `json:"category_type"`
Name string `json:"name"`
}
// Dashboard represents a dashboard definition for cloud integration.
type Dashboard struct {
Id string `json:"id"`
Url string `json:"url"`
Title string `json:"title"`
Description string `json:"description"`
Image string `json:"image"`
Definition *dashboardtypes.StorableDashboardData `json:"definition,omitempty"`
}
// UTILS
// GetCloudIntegrationDashboardID returns the dashboard id for a cloud integration, given the cloud provider, service id, and dashboard id.
// This is used to generate unique dashboard ids for cloud integration, and also to parse the dashboard id to get the cloud provider and service id when needed.
func GetCloudIntegrationDashboardID(cloudProvider CloudProviderType, svcId, dashboardId string) string {
return fmt.Sprintf("cloud-integration--%s--%s--%s", cloudProvider, svcId, dashboardId)
}
// GetDashboardsFromAssets returns the list of dashboards for the cloud provider service from definition
func GetDashboardsFromAssets(
svcId string,
orgID valuer.UUID,
cloudProvider CloudProviderType,
createdAt *time.Time,
assets Assets,
) []*dashboardtypes.Dashboard {
dashboards := make([]*dashboardtypes.Dashboard, 0)
for _, d := range assets.Dashboards {
author := fmt.Sprintf("%s-integration", cloudProvider)
dashboards = append(dashboards, &dashboardtypes.Dashboard{
ID: GetCloudIntegrationDashboardID(cloudProvider, svcId, d.Id),
Locked: true,
OrgID: orgID,
Data: *d.Definition,
TimeAuditable: types.TimeAuditable{
CreatedAt: *createdAt,
UpdatedAt: *createdAt,
},
UserAuditable: types.UserAuditable{
CreatedBy: author,
UpdatedBy: author,
},
})
}
return dashboards
}

View File

@@ -1,57 +0,0 @@
package cloudintegrationtypes
import (
"context"
"time"
"github.com/SigNoz/signoz/pkg/query-service/model"
"github.com/SigNoz/signoz/pkg/types"
)
type CloudIntegrationAccountStore interface {
ListConnected(ctx context.Context, orgId string, provider string) ([]types.CloudIntegration, *model.ApiError)
Get(ctx context.Context, orgId string, provider string, id string) (*types.CloudIntegration, *model.ApiError)
GetConnectedCloudAccount(ctx context.Context, orgId string, provider string, accountID string) (*types.CloudIntegration, *model.ApiError)
// Insert an account or update it by (cloudProvider, id)
// for specified non-empty fields
Upsert(
ctx context.Context,
orgId string,
provider string,
id *string,
config *types.AccountConfig,
accountId *string,
agentReport *types.AgentReport,
removedAt *time.Time,
) (*types.CloudIntegration, *model.ApiError)
}
type CloudIntegrationServiceStore interface {
Get(
ctx context.Context,
orgID string,
cloudAccountId string,
serviceType string,
) (*types.CloudServiceConfig, *model.ApiError)
Upsert(
ctx context.Context,
orgID string,
cloudProvider string,
cloudAccountId string,
serviceId string,
config types.CloudServiceConfig,
) (*types.CloudServiceConfig, *model.ApiError)
GetAllForAccount(
ctx context.Context,
orgID string,
cloudAccountId string,
) (
configsBySvcId map[string]*types.CloudServiceConfig,
apiErr *model.ApiError,
)
}

View File

@@ -1,4 +1,4 @@
package types
package integrationtypes
import (
"database/sql/driver"
@@ -6,6 +6,7 @@ import (
"time"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types"
"github.com/uptrace/bun"
)
@@ -26,17 +27,17 @@ var AllIntegrationUserEmails = []IntegrationUserEmail{
type InstalledIntegration struct {
bun.BaseModel `bun:"table:installed_integration"`
Identifiable
types.Identifiable
Type string `json:"type" bun:"type,type:text,unique:org_id_type"`
Config InstalledIntegrationConfig `json:"config" bun:"config,type:text"`
InstalledAt time.Time `json:"installed_at" bun:"installed_at,default:current_timestamp"`
OrgID string `json:"org_id" bun:"org_id,type:text,unique:org_id_type,references:organizations(id),on_delete:cascade"`
}
type InstalledIntegrationConfig map[string]interface{}
type InstalledIntegrationConfig map[string]any
// For serializing from db
func (c *InstalledIntegrationConfig) Scan(src interface{}) error {
func (c *InstalledIntegrationConfig) Scan(src any) error {
var data []byte
switch v := src.(type) {
case []byte:
@@ -67,8 +68,8 @@ func (c *InstalledIntegrationConfig) Value() (driver.Value, error) {
type CloudIntegration struct {
bun.BaseModel `bun:"table:cloud_integration"`
Identifiable
TimeAuditable
types.Identifiable
types.TimeAuditable
Provider string `json:"provider" bun:"provider,type:text,unique:provider_id"`
Config *AccountConfig `json:"config" bun:"config,type:text"`
AccountID *string `json:"account_id" bun:"account_id,type:text"`
@@ -194,8 +195,8 @@ func (r *AgentReport) Value() (driver.Value, error) {
type CloudIntegrationService struct {
bun.BaseModel `bun:"table:cloud_integration_service,alias:cis"`
Identifiable
TimeAuditable
types.Identifiable
types.TimeAuditable
Type string `bun:"type,type:text,notnull,unique:cloud_integration_id_type"`
Config CloudServiceConfig `bun:"config,type:text"`
CloudIntegrationID string `bun:"cloud_integration_id,type:text,notnull,unique:cloud_integration_id_type,references:cloud_integrations(id),on_delete:cascade"`

View File

@@ -1,17 +1,18 @@
package types
package savedviewtypes
import (
"strings"
"github.com/SigNoz/signoz/pkg/types"
"github.com/uptrace/bun"
)
type SavedView struct {
bun.BaseModel `bun:"table:saved_views"`
Identifiable
TimeAuditable
UserAuditable
types.Identifiable
types.TimeAuditable
types.UserAuditable
OrgID string `json:"orgId" bun:"org_id,notnull"`
Name string `json:"name" bun:"name,type:text,notnull"`
Category string `json:"category" bun:"category,type:text,notnull"`