Compare commits

...

15 Commits

Author SHA1 Message Date
Karan Balani
6ff63143bb Merge branch 'feat/meterreporter-jitter' of github.com:SigNoz/signoz into feat/meterreporter-jitter 2026-05-26 00:56:03 +05:30
Karan Balani
dcb235175e feat(meterreporter): emit delay_ns alongside delay string for graphing 2026-05-26 00:55:02 +05:30
Karan Balani
42af710df4 fix(meterreporter): log jitter delays as duration strings, not nanoseconds 2026-05-26 00:45:47 +05:30
Karan Balani
22f781e998 Merge branch 'main' into feat/meterreporter-jitter 2026-05-26 00:20:14 +05:30
Karan Balani
1e25fb263d refactor(meterreporter): drop redundant >=0 guards in jitter validation 2026-05-26 00:12:09 +05:30
Karan Balani
9693d2d9d3 fix(meterreporter): make jitter defaults track Interval via sentinel 2026-05-26 00:08:06 +05:30
Karan Balani
129e420c7b feat(meterreporter): log tick scheduling, reports, backfill and collector failures 2026-05-25 23:28:06 +05:30
Karan Balani
a71e43e1d1 feat(meterreporter): jitter first run and per-tick to spread Zeus load 2026-05-25 23:11:07 +05:30
Abhishek Kumar Singh
804ea2a7f8 feat: alert template processor + integration in notifiers (#10750)
Some checks are pending
build-staging / prepare (push) Waiting to run
build-staging / js-build (push) Blocked by required conditions
build-staging / go-build (push) Blocked by required conditions
build-staging / staging (push) Blocked by required conditions
Release Drafter / update_release_draft (push) Waiting to run
* chore: custom notifiers in alert manager

* chore: lint fixs

* chore: fix email linter

* chore: added tracing to msteamsv2 notifier

* feat: alert manager template to template title and notification body

* chore: updated test name + code for timeout errors

* chore: added utils for using variables with $ notation

* chore: exposed templates for alertmanager types

* feat: added preprocessor for alert templater

* chore: hooked preProcess function in expandTitle and body, added labels and annotations in alertdata

* chore: fix lint issues

* chore: added handling for missing variable used in template

* feat: converted alerttemplater to interface and updated tests

* refactor: added extractCommonKV instead of 2 different functions

* test: fix preprocessor test case

* feat: added support for  and  in templating

* chore: lint fix

* chore: renamed the interface

* chore: added test for missing function

* refactor: test case and sb related changed

* refactor: comments and test improvements

* chore: lint fix

* chore: updated comments

* feat: added basic html markdown templater

* chore: updated newline to markdown format

* feat: slack blockkit renderer using goldmark

* test: added test for html rendering

* feat: integrated slack blockit in markdownrenderer package and removed plaintext format

* chore: updated br with new line in test and logs added

* refactor: alert manager templater

* feat: added no-op formatter in markdown rederer

* chore: return missing variables as sorted list

* feat: alert notification processor

* chore: refactor notification processor and send processor in ReceiverIntegrations

* chore: return isDefaultTemplated true even in case of blank default template

* feat: updated email notifier

* feat: update ms team notifier with notification processor

* refactor: ms teams notifier

* chore: msteams note

* feat: added notification processor in opsgenie notifier

* feat: added notification processor in slack notifier

* feat: added notification processor in pagerduty notifier

* chore: added IsCustomTemplated helper function in result struct

* feat: added notification processor in webhook notifier

* chore: updated alertmanagernotify package with updated notifier signature

* feat: slack mrkdwn renderer

* feat: added new format in markdown renderer

* test: simplify TestRenderSlackMrkdwn

* test: add new test cases for Slack MRKDWN rendering

* feat: updated slack notifier with slack mrkdwn format

* fix: webhook notifier update annotations before preparing data

* fix: added handling for labels and annotations with `.` and `-`

* fix: handled <no value> in templated response

* test: added test in notification procesor for no value

* refactor: review comments

* refactor: lint fixes

* chore: updated licenses for notifiers

* chore: updated email notifier from upstream

* chore: lint fixes

* feat: added no value extension to render <no value> in html

* feat: email rendering with custom template in notification processor

* chore: integration of custom templating in rule manager

* chore: added action links to email and slack notifiers

* chore: fix linter and merge conflict issues

* feat: added `Literal` for CompareOperator and MatchType and expose from ruleManager

* chore: error logging + NoOp type definition

* feat: return single templating result from  with flag for template type

* fix: variables with symbols in template

* feat: slack mrkdwn renderer

* feat: custom raw html renderer to escape <no value>

* chore: integrated slack mrkdwn renderer and added NoOp formatter

* fix: email template directory for notification processor

* chore: remove static templates from pagerduty notifications

* chore: removed notifier test files

* fix: concurrent rendering in markdown renderer

* refactor: changes as per internal review

* chore: lint issue

* chore: removed special handling for softline break

* refactor: removed logger as markdown renderer dependency

* refactor: changed markdown renderer from interface to package-level functions

* refactor: changes as per internal review

* chore: removed notification processor

* chore: updated webhook notifier to send templated title and body in notification

* refactor: msteams skip logs and traces as factsset, slack code refactor

* chore: remove private annotations from pagerduty notifier

* chore: updated email template based on new template struct

* chore: update receiver integrations

* chore: outdated comment

* chore: move to templates/alertmanager

* chore: address comments

* chore: add example for templates

---------

Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
2026-05-25 17:07:55 +00:00
Jatinderjit Singh
a3a7fc4081 feat(planned-downtime): explicit toggle for all vs specific alert rules (#11272)
* feat(planned-downtime): explicit toggle for all vs specific alert rules

Replace the implicit "empty alert list silences everything" behavior
with a Radio toggle ("All alert rules" / "Specific alert rules") so
users can't accidentally silence every alert by forgetting to select
rules. The list view now displays an explicit "All alert rules" tag
instead of a dash for schedules that silence everything.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore: remove redundant messaging

* chore: reuse existing variable

* chore: fix typo

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
2026-05-25 16:54:46 +00:00
Vinicius Lourenço
3c8c318925 chore(upgrade-signoz): removed upgrade.signoz.io url (#11449) 2026-05-25 16:20:42 +00:00
Naman Verma
bb471848cc chore: feature flag for dashboard v2 (#11339)
* chore: feature flag for dashboard v2

* fix: fix alignment

* chore: add flag to v1 api as well
2026-05-25 14:08:04 +00:00
SagarRajput-7
bd55e70882 feat(boot-settings): runtime enable/disable control for PostHog and Appcues (#11416)
* feat(boot-settings): move SDK config from build-time env vars to runtime boot data injection

* feat(boot-settings): scope runtime injection to posthog/appcues enabled flags only

* feat(boot-settings): refactor code

* feat(boot-settings): refactor code

* feat(boot-settings): use generated WebSettings types for BE↔FE contract
2026-05-25 13:49:55 +00:00
Jatinderjit Singh
6cf22e98dd feat(planned-downtime): scope maintenance windows to label expressions (#11186)
* add maintenanceMuteStage to move planned maintenance to alertmanager

Rules previously skipped rule.Eval() entirely during maintenance windows.
This change moves suppression to MaintenanceMuter, injected as a Stage
in the alertmanager notification pipeline. Now rules always evaluate and
everys suppression is handled by alertmanager.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor: wrap routing pipeline once instead of per-route injection

Replace the per-route-entry loop with a single MultiStage wrap so
maintenance suppression runs once per dispatch group before routing.

* refactor: move maintenance mute stage into custom pipelineBuilder

Copy notify.PipelineBuilder locally so we can inject mms between the
silence stage and the receiver stage (GossipSettle → Inhibit →
TimeActive → TimeMute → Silence → mms → Receiver), matching the
correct suppression order the team requires.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore: add license header to pipeline_builder.go

Copied code originates from Apache-2.0 licensed Prometheus Alertmanager;
add dual copyright + SPDX identifier following the repo's convention.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore: replace SPDX tag with full Apache 2.0 license boilerplate

The full license text is unambiguously compliant with Apache 2.0 Section 4(a),
which requires giving recipients "a copy of this License".

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor: pass MaintenanceMuter directly to pipelineBuilder

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor: remove dead orgID param from task constructors

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* rename buildReceiverStage -> createReceiverStage

* refactor: replace maintenanceMuteStage with notify.NewMuteStage

MaintenanceMuter already satisfies types.Muter, and pipelineBuilder has
its own pb.metrics, so the hand-rolled maintenanceMuteStage wrapper is
redundant. Use notify.NewMuteStage(pb.muter, pb.metrics) directly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor: hoist MuteStage construction out of the receiver loop

MuteStage holds no per-receiver state, so one instance shared across
all receivers is sufficient — matching how is/ss are handled upstream.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor: always initialize maintenanceStore; remove nil guards

Tests now use a real sqlrulestore-backed MaintenanceMuter instead of
passing nil. With nil no longer a valid input, remove the nil guards
in server.go and pipeline_builder.go.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor: move MaintenanceMuter to Server and pass it to pipelineBuilder.New

- Remove muter from pipelineBuilder struct and newPipelineBuilder();
  pass it as a parameter to New() instead, consistent with inhibitor/silencer
- Store muter on Server so GetAlerts can call Mutes() alongside the
  inhibitor and silencer, ensuring maintenance-suppressed alerts show
  the correct muted status in API responses

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* remove redundant MemMarker wrapper

* feat: surface maintenance-suppressed alerts via mutedBy in GetAlerts

Alerts suppressed by an active maintenance window were being correctly
muted in the notification pipeline but appeared as state=active in the
v2 GetAlerts response, since MaintenanceMuter.Mutes had no marker
side-effect (unlike inhibitor/silencer).

Add MaintenanceMuter.MutedBy returning the matching window IDs, and
plumb a mutedByFunc callback through NewGettableAlertsFromAlertProvider
into AlertToOpenAPIAlert. The upstream v2 API forces state=suppressed
when mutedBy is non-empty, so the frontend's existing state-based
rendering picks it up without further changes.

Use the dedicated mutedBy field rather than SilencedBy to avoid
violating the "complete set of silence IDs" contract that anything
querying silences by ID would rely on.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* code cleanup

* refactor: move maintenance (planned downtime) to alertmanager packages

Types move from pkg/types/ruletypes/ to pkg/types/alertmanagertypes/:
- maintenance.go, recurrence.go, schedule.go (+ tests)

Store impl moves from pkg/ruler/rulestore/sqlrulestore/ to
pkg/alertmanager/alertmanagerstore/sqlalertmanagerstore/.

Maintenance windows mute alerts, so they belong with alertmanager
rather than the rule types.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test: add unit tests for MaintenanceMuter

Covers Mutes/MutedBy semantics (empty label, rule match, empty-RuleIDs
matches-all, future windows, multi-window) and the result cache
(single-fetch within TTL, stale-cache fallback on store error,
re-fetch after expiry, concurrency safety).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Update schema changes

* Re-add marker

* fix NewMaintenanceStore in tests

* Go lint fixes

* test: use mockery-generated mock for MaintenanceStore in muter tests

Replace hand-written fakeMaintenanceStore with a mockery-generated
MockMaintenanceStore, consistent with the alertmanagertest pattern.
Also adds MaintenanceStore to .mockery.yml so the mock stays in sync.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore: regenerate mocks via make gen-mocks

Picks up new MockHandler for the Handler interface in pkg/alertmanager
and regenerates MockMaintenanceStore with canonical mockery formatting.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* cleanup test

* test: add e2e muting tests for maintenance window behaviour

* Add label expression support to planned downtime

Alert instances can now be scoped by label expression
(e.g. env == "prod"), scoping suppression below the rule level.
A window with no rule IDs and a label expression silences any
alert whose labels match, regardless of which rule fired it;
when rule IDs are also present, the expression is evaluated only
within the matched rules.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Remove redundant || undefined from labelExpression assignment

* Move label expression evaluation into ShouldSkip

ShouldSkip now owns all three suppression checks in sequence:
rule ID match → schedule active → label expression.
IsActive passes nil labels so the expression check is skipped
(no instance labels available for UI status).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* remove redundant LabelSet->map conversion

* implement Down migration to drop label_expression column

* fix lset type and update openapi spec

* fix(tests): resolve envprovider env isolation, factory name length, and ShouldSkip signature

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* remove unused function `evaluateLabelExpression`

* chore: rename label expression to scope

* test(maintenance): add tests for scope label expression filtering

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test(e2e): verify scope-based maintenance muting in alertmanager flow

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Use `AND` instead of `&&`

Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>

* fix(maintenance): consolidate label-set-to-env conversion to avoid expr panic

Move ConvertLabelSetToEnv to alertmanagertypes so both the maintenance
scope evaluator and the route-policy evaluator share one implementation.
Dotted label keys (e.g. kubernetes.node) are expanded into nested maps,
preventing the expr-lang panic that occurs when one key is a prefix of
another.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore(planned-downtime): use clickable learn more link in scope tooltip

* chore(sqlmigration): renumber add_scope_to_planned_maintenance to 086

078 collided with add_sa_managed_role_txn; bumped to the next free number
and reordered registration after add_source_to_dashboard (085).

* refactor(maintenance): extract scope expression eval and surface errors

- Move ConvertLabelSetToEnv and EvalScopeExpression into expression.go
  with companion tests in expression_test.go.
- EvalScopeExpression now returns (bool, error) instead of swallowing
  compile/run failures and non-bool outputs; ShouldSkip logs the error
  via slog.Default() and falls back to not suppressing (safety-first).
- Update test fixtures to the SQL-style operator form (`=`, AND, OR)
  matching the placeholder and reviewer suggestions.

* chore: use `=` instead of `==` in expressions

* fix(maintenance): satisfy forbidigo/sloglint in scope eval

- Replace fmt.Errorf with pkg/errors Wrapf/Newf using a new
  ErrCodeInvalidScopeExpression code.
- Use slog ErrorContext (with context.Background()) instead of Error to
  satisfy sloglint.

* perf(maintenance): fold prefix-conflict detection into ConvertLabelSetToEnv

ConvertLabelSetToEnv now returns (env, conflict). The rulebased provider
drops its O(n^2) pre-scan and logs based on the flag, restoring the
previous O(n*d) cost while keeping the shared helper.

* chore: add docs URL for invalid scope

* refactor: don't log in types package

* remove down migration

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
2026-05-25 13:40:06 +00:00
Ashwin Bhatkal
39957d322f fix(planned-downtime): remove unused timezone dep from useMemo hooks (#11448)
The startTimeText and endTimeText useMemo hooks did not reference
timezone in their callback bodies, so including it in the dependency
arrays caused unnecessary recomputations whenever the timezone form
field changed.
2026-05-25 13:24:03 +00:00
66 changed files with 2583 additions and 500 deletions

View File

@@ -11,7 +11,7 @@ RUN apk update && \
COPY ./target/${OS}-${TARGETARCH}/signoz-community /root/signoz
COPY ./templates/email /root/templates
COPY ./templates /root/templates
COPY frontend/build/ /etc/signoz/web/
RUN chmod 755 /root /root/signoz

View File

@@ -12,7 +12,7 @@ RUN apk update && \
rm -rf /var/cache/apk/*
COPY ./target/${OS}-${ARCH}/signoz-community /root/signoz-community
COPY ./templates/email /root/templates
COPY ./templates /root/templates
COPY frontend/build/ /etc/signoz/web/
RUN chmod 755 /root /root/signoz-community

View File

@@ -11,7 +11,7 @@ RUN apk update && \
COPY ./target/${OS}-${TARGETARCH}/signoz /root/signoz
COPY ./templates/email /root/templates
COPY ./templates /root/templates
COPY frontend/build/ /etc/signoz/web/
RUN chmod 755 /root /root/signoz

View File

@@ -26,7 +26,7 @@ RUN go mod download
COPY ./cmd/ ./cmd/
COPY ./ee/ ./ee/
COPY ./pkg/ ./pkg/
COPY ./templates/email /root/templates
COPY ./templates /root/templates
COPY Makefile Makefile
RUN TARGET_DIR=/root ARCHS=${TARGETARCH} ZEUS_URL=${ZEUSURL} LICENSE_URL=${ZEUSURL}/api/v1 make go-build-enterprise-race

View File

@@ -12,7 +12,7 @@ RUN apk update && \
rm -rf /var/cache/apk/*
COPY ./target/${OS}-${ARCH}/signoz /root/signoz
COPY ./templates/email /root/templates
COPY ./templates /root/templates
COPY frontend/build/ /etc/signoz/web/
RUN chmod 755 /root /root/signoz

View File

@@ -35,7 +35,7 @@ RUN go mod download
COPY ./cmd/ ./cmd/
COPY ./ee/ ./ee/
COPY ./pkg/ ./pkg/
COPY ./templates/email /root/templates
COPY ./templates /root/templates
COPY Makefile Makefile
RUN TARGET_DIR=/root ARCHS=${TARGETARCH} ZEUS_URL=${ZEUSURL} LICENSE_URL=${ZEUSURL}/api/v1 make go-build-enterprise-race

View File

@@ -182,6 +182,11 @@ alertmanager:
poll_interval: 1m
# The URL under which Alertmanager is externally reachable (for example, if Alertmanager is served via a reverse proxy). Used for generating relative and absolute links back to Alertmanager itself.
external_url: http://localhost:8080
# The list of globs from which SigNoz's alertmanager notification templates are loaded (e.g. the email.signoz.html layout).
# This mirrors the upstream alertmanager `templates` config option. The upstream default templates (default.tmpl, email.tmpl)
# are always loaded from the embedded alertmanager assets, so only SigNoz's own templates need to be listed here.
templates:
- /opt/signoz/conf/templates/alertmanager/*.gotmpl
# The global configuration for the alertmanager. All the exahustive fields can be found in the upstream: https://github.com/prometheus/alertmanager/blob/efa05feffd644ba4accb526e98a8c6545d26a783/config/config.go#L833
global:
# ResolveTimeout is the time after which an alert is declared resolved if it has not been updated.

View File

@@ -129,6 +129,8 @@ components:
type: string
schedule:
$ref: '#/components/schemas/AlertmanagertypesSchedule'
scope:
type: string
status:
$ref: '#/components/schemas/AlertmanagertypesMaintenanceStatus'
updatedAt:
@@ -272,6 +274,8 @@ components:
type: string
schedule:
$ref: '#/components/schemas/AlertmanagertypesSchedule'
scope:
type: string
required:
- name
- schedule

View File

@@ -5,6 +5,7 @@ import (
"encoding/json"
"fmt"
"log/slog"
"math/rand/v2"
"time"
"github.com/SigNoz/signoz/pkg/errors"
@@ -94,21 +95,33 @@ func newProvider(
func (provider *Provider) Start(ctx context.Context) error {
close(provider.healthyC)
provider.collect(ctx)
startDelay := jitter(provider.config.ResolvedMaxStartJitter())
provider.settings.Logger().InfoContext(ctx, "scheduling first meter collect", slog.String("delay", startDelay.String()), slog.Int64("delay_ns", startDelay.Nanoseconds()))
ticker := time.NewTicker(provider.config.Interval)
defer ticker.Stop()
timer := time.NewTimer(startDelay)
defer timer.Stop()
for {
select {
case <-provider.stopC:
return nil
case <-ticker.C:
case <-timer.C:
provider.collect(ctx)
next := provider.config.Interval - jitter(provider.config.ResolvedMaxTickJitter())
timer.Reset(next)
provider.settings.Logger().InfoContext(ctx, "scheduled next meter collect", slog.String("delay", next.String()), slog.Int64("delay_ns", next.Nanoseconds()))
}
}
}
// jitter returns a uniform random duration in [0, max). Returns 0 if max <= 0.
func jitter(max time.Duration) time.Duration {
if max <= 0 {
return 0
}
return time.Duration(rand.Int64N(int64(max)))
}
func (provider *Provider) collect(ctx context.Context) {
ctx, span := provider.settings.Tracer().Start(ctx, "meterreporter.Collect", trace.WithAttributes(attribute.String("meterreporter.provider", "http")))
defer span.End()
@@ -147,6 +160,7 @@ func (provider *Provider) collect(ctx context.Context) {
}
func (provider *Provider) Stop(ctx context.Context) error {
provider.settings.Logger().InfoContext(ctx, "stopping meter reporter")
close(provider.stopC)
return nil
}
@@ -170,6 +184,11 @@ func (provider *Provider) collectOrg(ctx context.Context, org *types.Organizatio
start, end, ok := backfillRange(nextByCollector, todayStart)
if ok {
provider.settings.Logger().InfoContext(ctx, "backfilling sealed days",
slog.String("org_id", org.ID.StringValue()),
slog.String("start", start.Format("2006-01-02")),
slog.String("end", end.Format("2006-01-02")))
for day := start; !day.After(end); day = day.AddDate(0, 0, 1) {
eligible := eligibleCollectors(provider.collectorsByName, nextByCollector, day)
if len(eligible) == 0 {
@@ -257,6 +276,10 @@ func (provider *Provider) report(ctx context.Context, orgID valuer.UUID, license
collectedReadings, err := collector.Collect(ctx, orgID, license, window)
if err != nil {
provider.metrics.collections.Add(ctx, 1, metric.WithAttributes(meterAttr, errors.TypeAttr(err)))
provider.settings.Logger().ErrorContext(ctx, "meter collector failed",
errors.Attr(err),
slog.String("org_id", orgID.StringValue()),
slog.String("meter", collector.Name().String()))
continue
}
@@ -283,6 +306,10 @@ func (provider *Provider) report(ctx context.Context, orgID valuer.UUID, license
provider.metrics.reports.Add(ctx, 1)
provider.metrics.meters.Add(ctx, int64(len(meters)))
provider.settings.Logger().InfoContext(ctx, "reported meters to zeus",
slog.String("org_id", orgID.StringValue()),
slog.String("date", date),
slog.Int("meters", len(meters)))
return nil
}

View File

@@ -89,6 +89,15 @@ func (ah *APIHandler) getFeatureFlags(w http.ResponseWriter, r *http.Request) {
Route: "",
})
useDashboardV2 := ah.Signoz.Flagger.BooleanOrEmpty(ctx, flagger.FeatureUseDashboardV2, evalCtx)
featureSet = append(featureSet, &licensetypes.Feature{
Name: valuer.NewString(flagger.FeatureUseDashboardV2.String()),
Active: useDashboardV2,
Usage: 0,
UsageLimit: -1,
Route: "",
})
if constants.IsDotMetricsEnabled {
for idx, feature := range featureSet {
if feature.Name == licensetypes.DotMetricsEnabled {

View File

@@ -94,6 +94,19 @@
}
})();
</script>
<script type="application/json" id="signoz-boot-settings">
[[.Settings]]
</script>
<script>
try {
var _el = document.getElementById('signoz-boot-settings');
window.signozBootData = {
settings: _el ? JSON.parse(_el.textContent) : null,
};
} catch (e) {
window.signozBootData = { settings: null };
}
</script>
<noscript>You need to enable JavaScript to run this app.</noscript>
<div id="root"></div>
@@ -135,7 +148,10 @@
</script>
<script>
var APPCUES_APP_ID = '<%- APPCUES_APP_ID %>';
if (APPCUES_APP_ID) {
var appcuesSettings =
((window.signozBootData || {}).settings || {}).appcues || {};
var appcuesEnabled = appcuesSettings.enabled !== false;
if (APPCUES_APP_ID && appcuesEnabled) {
(function (d, t) {
var a = d.createElement(t);
a.async = 1;

View File

@@ -35,6 +35,7 @@ import { PreferenceContextProvider } from 'providers/preferences/context/Prefere
import { QueryBuilderProvider } from 'providers/QueryBuilder';
import { LicenseStatus } from 'types/api/licensesV3/getActive';
import { extractDomain } from 'utils/app';
import { bootSettings } from 'utils/bootData';
import { Home } from './pageComponents';
import PrivateRoute from './Private';
@@ -332,7 +333,7 @@ function App(): JSX.Element {
useEffect(() => {
if (isCloudUser || isEnterpriseSelfHostedUser) {
if (process.env.POSTHOG_KEY) {
if (bootSettings.posthog.enabled && process.env.POSTHOG_KEY) {
posthog.init(process.env.POSTHOG_KEY, {
api_host: 'https://us.i.posthog.com',
person_profiles: 'identified_only', // or 'always' to create profiles for anonymous users as well

View File

@@ -225,6 +225,10 @@ export interface AlertmanagertypesPlannedMaintenanceDTO {
*/
name: string;
schedule: AlertmanagertypesScheduleDTO;
/**
* @type string
*/
scope?: string;
status: AlertmanagertypesMaintenanceStatusDTO;
/**
* @type string
@@ -1714,6 +1718,10 @@ export interface AlertmanagertypesPostablePlannedMaintenanceDTO {
*/
name: string;
schedule: AlertmanagertypesScheduleDTO;
/**
* @type string
*/
scope?: string;
}
export interface AlertmanagertypesPostableRoutePolicyDTO {

View File

@@ -10,9 +10,6 @@ export const DEFAULT_AUTH0_APP_REDIRECTION_PATH = ROUTES.APPLICATION;
export const INVITE_MEMBERS_HASH = '#invite-team-members';
export const SIGNOZ_UPGRADE_PLAN_URL =
'https://upgrade.signoz.io/upgrade-from-app';
export const DASHBOARD_TIME_IN_DURATION = 'refreshInterval';
export const DEFAULT_ENTITY_VERSION = 'v3';

View File

@@ -11,4 +11,5 @@ export enum FeatureKeys {
DOT_METRICS_ENABLED = 'dot_metrics_enabled',
USE_JSON_BODY = 'use_json_body',
USE_FINE_GRAINED_AUTHZ = 'use_fine_grained_authz',
USE_DASHBOARD_V2 = 'use_dashboard_v2',
}

View File

@@ -1,4 +1,3 @@
import { SIGNOZ_UPGRADE_PLAN_URL } from 'constants/app';
import CreateAlertChannels from 'container/CreateAlertChannels';
import { ChannelType } from 'container/CreateAlertChannels/config';
import {
@@ -313,16 +312,6 @@ describe('Create Alert Channel (Normal User)', () => {
expect(screen.getByText('Microsoft Teams')).toBeInTheDocument();
});
it.skip('Should check if the upgrade plan message is shown', () => {
expect(screen.getByText('Upgrade to a Paid Plan')).toBeInTheDocument();
expect(
screen.getByText(/This feature is available for paid plans only./),
).toBeInTheDocument();
const link = screen.getByRole('link', { name: 'Click here' });
expect(link).toBeInTheDocument();
expect(link).toHaveAttribute('href', SIGNOZ_UPGRADE_PLAN_URL);
expect(screen.getByText(/to Upgrade/)).toBeInTheDocument();
});
it('Should check if the form buttons are displayed properly (Save, Test, Back)', () => {
expect(
screen.getByRole('button', { name: 'button_save_channel' }),

View File

@@ -81,6 +81,20 @@
}
}
.alert-rule-scope {
margin-bottom: 12px;
.ant-radio-wrapper {
color: var(--l1-foreground);
}
}
.alert-rule-all-warning {
font-size: 12px;
font-weight: 400;
color: var(--l2-foreground);
}
.formItemWithBullet {
margin-bottom: 0;
}

View File

@@ -1,5 +1,5 @@
import React, { useCallback, useEffect, useMemo, useState } from 'react';
import { Check } from '@signozhq/icons';
import { Check, Info } from '@signozhq/icons';
import {
Button,
DatePicker,
@@ -8,9 +8,11 @@ import {
FormInstance,
Input,
Modal,
Radio,
Select,
SelectProps,
Spin,
Tooltip,
} from 'antd';
import { Typography } from '@signozhq/ui/typography';
import type { DefaultOptionType } from 'antd/es/select';
@@ -70,14 +72,18 @@ const TZ_OPTIONS: DefaultOptionType[] = ALL_TIME_ZONES.map(
}),
);
type AlertRuleScope = 'all' | 'specific';
interface PlannedDowntimeFormData {
name: string;
startTime: dayjs.Dayjs | null;
endTime: dayjs.Dayjs | null;
recurrence?: AlertmanagertypesRecurrenceDTO;
alertRuleScope: AlertRuleScope;
alertRules: DefaultOptionType[];
recurrenceSelect?: AlertmanagertypesRecurrenceDTO;
timezone?: string;
scope?: string;
}
const customFormat = DATE_TIME_FORMATS.ORDINAL_DATETIME;
@@ -127,6 +133,12 @@ export function PlannedDowntimeForm(
recurrenceOptions.doesNotRepeat.value,
);
const [alertRuleScope, setAlertRuleScope] = useState<AlertRuleScope>(
initialValues.id && (initialValues.alertIds || []).length === 0
? 'all'
: 'specific',
);
const { notifications } = useNotifications();
const { showErrorModal } = useErrorModal();
@@ -140,10 +152,14 @@ export function PlannedDowntimeForm(
const saveHandler = useCallback(
async (values: PlannedDowntimeFormData) => {
const data: AlertmanagertypesPostablePlannedMaintenanceDTO = {
alertIds: values.alertRules
.map((alert) => alert.value)
.filter((alert) => alert !== undefined) as string[],
alertIds:
values.alertRuleScope === 'all'
? []
: (values.alertRules
.map((alert) => alert.value)
.filter((alert) => alert !== undefined) as string[]),
name: values.name,
scope: values.scope,
schedule: {
startTime: values.startTime?.format(),
endTime: values.endTime?.format(),
@@ -262,12 +278,13 @@ export function PlannedDowntimeForm(
const startTime = schedule?.recurrence?.startTime || schedule?.startTime;
const endTime = schedule?.recurrence?.endTime || schedule?.endTime;
const initialAlertIds = initialValues.alertIds || [];
return {
name: defaultTo(initialValues.name, ''),
alertRules: getAlertOptionsFromIds(
initialValues.alertIds || [],
alertOptions,
),
alertRuleScope:
isEditMode && initialAlertIds.length === 0 ? 'all' : 'specific',
alertRules: getAlertOptionsFromIds(initialAlertIds, alertOptions),
startTime: startTime ? dayjs(startTime).tz(schedule.timezone) : null,
endTime: endTime ? dayjs(endTime).tz(schedule.timezone) : null,
recurrence: {
@@ -278,11 +295,13 @@ export function PlannedDowntimeForm(
duration: getDurationInfo(schedule?.recurrence?.duration)?.value ?? '',
} as AlertmanagertypesRecurrenceDTO,
timezone: schedule?.timezone as string,
scope: initialValues.scope || '',
};
}, [initialValues, alertOptions]);
useEffect(() => {
setSelectedTags(formattedInitialValues.alertRules);
setAlertRuleScope(formattedInitialValues.alertRuleScope);
form.setFieldsValue({ ...formattedInitialValues });
}, [form, formattedInitialValues, initialValues]);
@@ -311,7 +330,7 @@ export function PlannedDowntimeForm(
default:
return `Scheduled for ${formattedStartDate} starting at ${formattedStartTime}.`;
}
}, [formData, recurrenceType, timezone]);
}, [formData, recurrenceType]);
const endTimeText = useMemo((): string => {
const endTime = formData.endTime;
@@ -322,7 +341,7 @@ export function PlannedDowntimeForm(
const formattedEndTime = endTime.format(TIME_FORMAT);
const formattedEndDate = endTime.format(DATE_FORMAT);
return `Scheduled to end maintenance on ${formattedEndDate} at ${formattedEndTime}.`;
}, [formData, recurrenceType, timezone]);
}, [formData, recurrenceType]);
return (
<Modal
@@ -345,6 +364,7 @@ export function PlannedDowntimeForm(
onFinish={onFinish}
onValuesChange={(): void => {
setRecurrenceType(form.getFieldValue('recurrence')?.repeatType as string);
setAlertRuleScope(form.getFieldValue('alertRuleScope') as AlertRuleScope);
handleFormData(form.getFieldsValue());
}}
autoComplete="off"
@@ -444,50 +464,107 @@ export function PlannedDowntimeForm(
<div className="scheduleTimeInfoText">{endTimeText}</div>
)}
<div>
<div className="alert-rule-form">
<Typography style={{ marginBottom: 8 }}>Silence Alerts</Typography>
<Typography style={{ marginBottom: 8 }} className="alert-rule-info">
(Leave empty to silence all alerts)
</Typography>
</div>
<Form.Item noStyle shouldUpdate>
<AlertRuleTags
closable
selectedTags={selectedTags}
handleClose={handleClose}
/>
</Form.Item>
<Form.Item name={alertRuleFormName}>
<Select
placeholder="Search for alerts rules or groups..."
mode="multiple"
status={isError ? 'error' : undefined}
loading={isLoading}
tagRender={noTagRenderer}
onChange={handleAlertRulesChange}
showSearch
options={alertOptions}
filterOption={(input, option): boolean =>
(option?.label as string)?.toLowerCase()?.includes(input.toLowerCase())
}
notFoundContent={
isLoading ? (
<span>
<Spin size="small" /> Loading...
</span>
) : (
<span>No alert available.</span>
)
}
>
{alertOptions?.map((option) => (
<Select.Option key={option.value} value={option.value}>
{option.label}
</Select.Option>
))}
</Select>
<Typography style={{ marginBottom: 8 }}>Silence Alerts</Typography>
<Form.Item
name="alertRuleScope"
initialValue="specific"
className="alert-rule-scope"
>
<Radio.Group>
<Radio value="all">All alert rules</Radio>
<Radio value="specific">Specific alert rules</Radio>
</Radio.Group>
</Form.Item>
{alertRuleScope === 'specific' && (
<>
<Form.Item noStyle shouldUpdate>
<AlertRuleTags
closable
selectedTags={selectedTags}
handleClose={handleClose}
/>
</Form.Item>
<Form.Item
name={alertRuleFormName}
rules={[
{
validator: async (
_rule,
value: DefaultOptionType[] | undefined,
): Promise<void> => {
if (!value || value.length === 0) {
throw new Error(
'Select at least one alert rule, or choose "All alert rules" to silence everything.',
);
}
},
},
]}
>
<Select
placeholder="Search for alert rules or groups..."
mode="multiple"
status={isError ? 'error' : undefined}
loading={isLoading}
tagRender={noTagRenderer}
onChange={handleAlertRulesChange}
showSearch
options={alertOptions}
filterOption={(input, option): boolean =>
(option?.label as string)
?.toLowerCase()
?.includes(input.toLowerCase())
}
notFoundContent={
isLoading ? (
<span>
<Spin size="small" /> Loading...
</span>
) : (
<span>No alert available.</span>
)
}
>
{alertOptions?.map((option) => (
<Select.Option key={option.value} value={option.value}>
{option.label}
</Select.Option>
))}
</Select>
</Form.Item>
</>
)}
</div>
<Form.Item
label={
<span>
Scope&nbsp;
<Tooltip
mouseLeaveDelay={0.3}
title={
<span>
Scope the planned downtime by alert labels.{' '}
<a
href="https://signoz.io/docs/alerts-management/planned-maintenance/#scoping-with-label-expressions"
target="_blank"
rel="noopener noreferrer"
>
Learn more
</a>
</span>
}
>
<Info size={13} />
</Tooltip>
</span>
}
name="scope"
>
<Input.TextArea
placeholder='e.g. env = "prod" AND region = "us-east-1"'
autoSize={{ minRows: 2, maxRows: 4 }}
/>
</Form.Item>
<Form.Item style={{ marginBottom: 0 }}>
<ModalButtonWrapper>
<Button

View File

@@ -204,7 +204,7 @@ export function CollapseListContent({
selectedTags={alertOptions}
/>
) : (
'-'
<Tag className="all-alerts-tag">All alert rules</Tag>
),
)}
</Flex>

View File

@@ -1,5 +1,6 @@
// eslint-disable-next-line no-restricted-imports
import { compose, Store } from 'redux';
import type { WebSettings } from 'types/generated/webSettings';
declare global {
interface Window {
@@ -7,6 +8,7 @@ declare global {
pylon: any;
Appcues: Record<string, any>;
__REDUX_DEVTOOLS_EXTENSION_COMPOSE__: typeof compose;
signozBootData?: { settings: WebSettings | null };
}
}

View File

@@ -0,0 +1,74 @@
export {};
type BootData = typeof import('../bootData');
function loadModule(settings?: object | null): BootData {
(window as any).signozBootData =
settings !== undefined ? { settings } : undefined;
let mod!: BootData;
jest.isolateModules(() => {
// oxlint-disable-next-line typescript-eslint/no-require-imports, typescript-eslint/no-var-requires
mod = require('../bootData');
});
return mod;
}
afterEach(() => {
delete (window as any).signozBootData;
});
describe('when window.signozBootData is absent', () => {
it('defaults posthog and appcues to enabled', () => {
const { bootSettings } = loadModule();
expect(bootSettings.posthog.enabled).toBe(true);
expect(bootSettings.appcues.enabled).toBe(true);
});
});
describe('when window.signozBootData.settings is null (injection failed)', () => {
it('defaults posthog and appcues to enabled', () => {
const { bootSettings } = loadModule(null);
expect(bootSettings.posthog.enabled).toBe(true);
expect(bootSettings.appcues.enabled).toBe(true);
});
});
describe('when window.signozBootData.settings is populated', () => {
it('reads posthog enabled: true', () => {
const { bootSettings } = loadModule({ posthog: { enabled: true } });
expect(bootSettings.posthog.enabled).toBe(true);
});
it('reads posthog enabled: false', () => {
const { bootSettings } = loadModule({ posthog: { enabled: false } });
expect(bootSettings.posthog.enabled).toBe(false);
});
it('reads appcues enabled: true', () => {
const { bootSettings } = loadModule({ appcues: { enabled: true } });
expect(bootSettings.appcues.enabled).toBe(true);
});
it('reads appcues enabled: false', () => {
const { bootSettings } = loadModule({ appcues: { enabled: false } });
expect(bootSettings.appcues.enabled).toBe(false);
});
it('missing sub-namespace defaults to enabled', () => {
const { bootSettings } = loadModule({ posthog: { enabled: false } });
expect(bootSettings.appcues.enabled).toBe(true);
});
});
describe('when window.signozBootData exists but settings is undefined', () => {
it('defaults posthog and appcues to enabled', () => {
(window as any).signozBootData = {};
let mod!: BootData;
jest.isolateModules(() => {
// oxlint-disable-next-line typescript-eslint/no-require-imports, typescript-eslint/no-var-requires
mod = require('../bootData');
});
expect(mod.bootSettings.posthog.enabled).toBe(true);
expect(mod.bootSettings.appcues.enabled).toBe(true);
});
});

View File

@@ -0,0 +1,11 @@
import type { WebSettings } from 'types/generated/webSettings';
const raw = window.signozBootData?.settings as
| Partial<WebSettings>
| null
| undefined;
export const bootSettings: Readonly<WebSettings> = {
posthog: { enabled: raw?.posthog?.enabled ?? true },
appcues: { enabled: raw?.appcues?.enabled ?? true },
};

View File

@@ -23,6 +23,20 @@ function devBasePathPlugin(basePath: string): Plugin {
};
}
function devBootDataPlugin(env: Record<string, string>): Plugin {
return {
name: 'dev-boot-data',
apply: 'serve',
transformIndexHtml(html): string {
const settings = {
posthog: { enabled: env.VITE_POSTHOG_ENABLED !== 'false' },
appcues: { enabled: env.VITE_APPCUES_ENABLED !== 'false' },
};
return html.replaceAll('[[.Settings]]', JSON.stringify(settings));
},
};
}
function rawMarkdownPlugin(): Plugin {
return {
name: 'raw-markdown',
@@ -47,6 +61,7 @@ export default defineConfig(({ mode }): UserConfig => {
tsconfigPaths(),
rawMarkdownPlugin(),
devBasePathPlugin(basePath),
devBootDataPlugin(env),
react(),
createHtmlPlugin({
inject: {

View File

@@ -23,7 +23,13 @@ import (
"sync"
"time"
htmltemplate "html/template"
"github.com/SigNoz/signoz/pkg/alertmanager/alertmanagertemplate"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/templating/markdownrenderer"
"github.com/SigNoz/signoz/pkg/types/alertmanagertypes"
"github.com/SigNoz/signoz/pkg/types/ruletypes"
commoncfg "github.com/prometheus/common/config"
"github.com/prometheus/alertmanager/config"
@@ -34,20 +40,39 @@ import (
const (
Integration = "email"
// alertEmailLayoutTemplate is the name of the HTML layout template that
// wraps the rendered alert bodies. It is loaded into the notification
// template (n.tmpl) from the alertmanager templates config and lives at
// templates/alertmanager/email.gotmpl.
alertEmailLayoutTemplate = "email.signoz.html"
)
// Email implements a Notifier for email notifications.
type Email struct {
conf *config.EmailConfig
tmpl *template.Template
logger *slog.Logger
hostname string
conf *config.EmailConfig
tmpl *template.Template
logger *slog.Logger
hostname string
templater alertmanagertypes.Templater
}
// layoutData is the value passed to the email.signoz.html layout
// template. It embeds NotificationTemplateData so templates can reference
// `.Alert.Status`, `.Alert.TotalFiring`, `.Alert.TotalResolved`,
// `.NotificationTemplateData.ExternalURL`, etc. alongside the rendered
// Title and per-alert Bodies.
type layoutData struct {
alertmanagertypes.NotificationTemplateData
Title string
Bodies []htmltemplate.HTML
}
var errNoAuthUsernameConfigured = errors.NewInternalf(errors.CodeInternal, "no auth username configured")
// New returns a new Email notifier.
func New(c *config.EmailConfig, t *template.Template, l *slog.Logger) *Email {
// New returns a new Email notifier. When the email.signoz.html layout is
// not defined in t, custom-body alerts fall back to plain <div>-wrapped HTML.
func New(c *config.EmailConfig, t *template.Template, l *slog.Logger, templater alertmanagertypes.Templater) *Email {
if _, ok := c.Headers["Subject"]; !ok {
c.Headers["Subject"] = config.DefaultEmailSubject
}
@@ -63,7 +88,7 @@ func New(c *config.EmailConfig, t *template.Template, l *slog.Logger) *Email {
if err != nil {
h = "localhost.localdomain"
}
return &Email{conf: c, tmpl: t, logger: l, hostname: h}
return &Email{conf: c, tmpl: t, logger: l, hostname: h, templater: templater}
}
// auth resolves a string of authentication mechanisms.
@@ -199,9 +224,9 @@ func (n *Email) Notify(ctx context.Context, as ...*types.Alert) (bool, error) {
if ok, mech := c.Extension("AUTH"); ok {
auth, err := n.auth(mech)
if err != nil && err != errNoAuthUsernameConfigured {
if err != nil && !errors.Is(err, errNoAuthUsernameConfigured) {
return true, errors.WrapInternalf(err, errors.CodeInternal, "find auth mechanism")
} else if err == errNoAuthUsernameConfigured {
} else if errors.Is(err, errNoAuthUsernameConfigured) {
n.logger.DebugContext(ctx, "no auth username configured. Attempting to send email without authenticating")
}
if auth != nil {
@@ -245,6 +270,16 @@ func (n *Email) Notify(ctx context.Context, as ...*types.Alert) (bool, error) {
}
}
// Prepare the content for the email. subject, when non-empty, overrides
// the configured Subject header for this notification only. We deliberately
// do not mutate n.conf.Headers here: the config map is shared across
// concurrent notifications to the same receiver.
subject, htmlBody, err := n.prepareContent(ctx, as)
if err != nil {
n.logger.ErrorContext(ctx, "failed to prepare notification content", errors.Attr(err))
return false, err
}
// Send the email headers and body.
message, err := c.Data()
if err != nil {
@@ -262,6 +297,10 @@ func (n *Email) Notify(ctx context.Context, as ...*types.Alert) (bool, error) {
buffer := &bytes.Buffer{}
for header, t := range n.conf.Headers {
if header == "Subject" {
fmt.Fprintf(buffer, "%s: %s\r\n", header, mime.QEncoding.Encode("utf-8", subject))
continue
}
value, err := n.tmpl.ExecuteTextString(t, data)
if err != nil {
return false, errors.WrapInternalf(err, errors.CodeInternal, "execute %q header template", header)
@@ -336,7 +375,7 @@ func (n *Email) Notify(ctx context.Context, as ...*types.Alert) (bool, error) {
}
}
if len(n.conf.HTML) > 0 {
if htmlBody != "" {
// Html template
// Preferred alternative placed last per section 5.1.4 of RFC 2046
// https://www.ietf.org/rfc/rfc2046.txt
@@ -347,12 +386,8 @@ func (n *Email) Notify(ctx context.Context, as ...*types.Alert) (bool, error) {
if err != nil {
return false, errors.WrapInternalf(err, errors.CodeInternal, "create part for html template")
}
body, err := n.tmpl.ExecuteHTMLString(n.conf.HTML, data)
if err != nil {
return false, errors.WrapInternalf(err, errors.CodeInternal, "execute html template")
}
qw := quotedprintable.NewWriter(w)
_, err = qw.Write([]byte(body))
_, err = qw.Write([]byte(htmlBody))
if err != nil {
return true, errors.WrapInternalf(err, errors.CodeInternal, "write HTML part")
}
@@ -381,6 +416,124 @@ func (n *Email) Notify(ctx context.Context, as ...*types.Alert) (bool, error) {
return false, nil
}
// prepareContent returns a subject override (empty when the default config
// Subject should be used) and the HTML body for the email. Callers must treat
// the subject as local state and never write it back to n.conf.Headers.
func (n *Email) prepareContent(ctx context.Context, alerts []*types.Alert) (string, string, error) {
customTitle, customBody := alertmanagertemplate.ExtractTemplatesFromAnnotations(alerts)
result, err := n.templater.Expand(ctx, alertmanagertypes.ExpandRequest{
TitleTemplate: customTitle,
BodyTemplate: customBody,
DefaultTitleTemplate: n.conf.Headers["Subject"],
DefaultBodyTemplate: n.conf.HTML,
}, alerts)
if err != nil {
return "", "", err
}
subject := result.Title
if !result.IsDefaultBody {
// Custom-body path: render each expanded markdown body to HTML, then
// wrap the whole thing in the email.signoz.html layout (or fall
// back to plain <div> wrapping when the layout template is not loaded).
for i, body := range result.Body {
if body == "" {
continue
}
rendered, err := markdownrenderer.RenderHTML(body)
if err != nil {
return "", "", err
}
result.Body[i] = rendered
}
appendRelatedLinkButtons(alerts, result.Body)
html, err := n.renderLayout(result)
if err != nil {
n.logger.WarnContext(ctx, "custom email template rendering failed, falling back to plain <div> wrap", errors.Attr(err))
return subject, wrapBodiesAsDivs(result.Body), nil
}
return subject, html, nil
}
return subject, result.Body[0], nil
}
// renderLayout wraps result in the email.signoz.html HTML layout loaded
// into n.tmpl from the alertmanager templates config. Returns an error when the
// layout template is not defined (e.g. in tests where no templates are loaded)
// so prepareContent can fall back to plain <div> wrapping.
func (n *Email) renderLayout(result *alertmanagertypes.ExpandResult) (string, error) {
bodies := make([]htmltemplate.HTML, 0, len(result.Body))
for _, b := range result.Body {
bodies = append(bodies, htmltemplate.HTML(b))
}
data := layoutData{Title: result.Title, Bodies: bodies}
if result.NotificationData != nil {
data.NotificationTemplateData = *result.NotificationData
}
html, err := n.tmpl.ExecuteHTMLString(`{{ template "`+alertEmailLayoutTemplate+`" . }}`, data)
if err != nil {
return "", errors.WrapInternalf(err, errors.CodeInternal, "failed to render email layout")
}
return html, nil
}
// appendRelatedLinkButtons appends "View Related Logs/Traces" buttons to each
// per-alert body when the rule manager attached the corresponding annotation.
// bodies is positionally aligned with alerts (see alertmanagertemplate.Prepare);
// empty bodies are skipped so we never attach a button to an alert that produced
// no visible content.
func appendRelatedLinkButtons(alerts []*types.Alert, bodies []string) {
for i := range bodies {
if i >= len(alerts) || bodies[i] == "" {
continue
}
if link := alerts[i].Annotations[ruletypes.AnnotationRelatedLogs]; link != "" {
bodies[i] += htmlButton("View Related Logs", string(link))
}
if link := alerts[i].Annotations[ruletypes.AnnotationRelatedTraces]; link != "" {
bodies[i] += htmlButton("View Related Traces", string(link))
}
}
}
func wrapBodiesAsDivs(bodies []string) string {
var b strings.Builder
for _, part := range bodies {
if part == "" {
continue
}
b.WriteString("<div>")
b.WriteString(part)
b.WriteString("</div>")
}
return b.String()
}
func htmlButton(text, url string) string {
return fmt.Sprintf(`
<a href="%s" target="_blank" style="text-decoration: none;">
<button style="
padding: 6px 16px;
/* Default System Font */
font-family: sans-serif;
font-size: 14px;
font-weight: 500;
line-height: 1.5;
/* Light Theme & Dynamic Background (Solid) */
color: #111827;
background-color: #f9fafb;
/* Static Outline */
border: 1px solid #d1d5db;
border-radius: 4px;
cursor: pointer;
">
%s
</button>
</a>`, url, text)
}
type loginAuth struct {
username, password string
}

View File

@@ -8,6 +8,7 @@ import (
"context"
"fmt"
"io"
"log/slog"
"net"
"net/http"
"net/url"
@@ -17,7 +18,10 @@ import (
"testing"
"time"
"github.com/SigNoz/signoz/pkg/alertmanager/alertmanagertemplate"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types/alertmanagertypes"
"github.com/SigNoz/signoz/pkg/types/ruletypes"
"github.com/emersion/go-smtp"
commoncfg "github.com/prometheus/common/config"
"github.com/prometheus/common/model"
@@ -42,6 +46,11 @@ const (
emailFrom = "alertmanager@example.com"
)
// testTemplater returns a Templater bound to tmpl with a discard logger.
func testTemplater(tmpl *template.Template) alertmanagertypes.Templater {
return alertmanagertemplate.New(tmpl, slog.New(slog.DiscardHandler))
}
// email represents an email returned by the MailDev REST API.
// See https://github.com/djfarrelly/MailDev/blob/master/docs/rest.md.
type email struct {
@@ -162,7 +171,7 @@ func notifyEmailWithContext(ctx context.Context, t *testing.T, cfg *config.Email
return nil, false, err
}
email := New(cfg, tmpl, promslog.NewNopLogger())
email := New(cfg, tmpl, promslog.NewNopLogger(), testTemplater(tmpl))
retry, err := email.Notify(ctx, firingAlert)
if err != nil {
@@ -706,7 +715,7 @@ func TestEmailRejected(t *testing.T) {
tmpl, firingAlert, err := prepare(cfg)
require.NoError(t, err)
e := New(cfg, tmpl, promslog.NewNopLogger())
e := New(cfg, tmpl, promslog.NewNopLogger(), testTemplater(tmpl))
// Send the alert to mock SMTP server.
retry, err := e.Notify(context.Background(), firingAlert)
@@ -1030,6 +1039,135 @@ func TestEmailImplicitTLS(t *testing.T) {
}
}
func TestPrepareContent(t *testing.T) {
t.Run("default title template; custom body template", func(t *testing.T) {
tmpl, err := template.FromGlobs([]string{})
require.NoError(t, err)
tmpl.ExternalURL, _ = url.Parse("http://am")
bodyTpl := "line $labels.instance"
a1 := &types.Alert{
Alert: model.Alert{
Labels: model.LabelSet{
model.LabelName("instance"): model.LabelValue("one"),
},
Annotations: model.LabelSet{
model.LabelName(ruletypes.AnnotationBodyTemplate): model.LabelValue(bodyTpl),
},
},
}
a2 := &types.Alert{
Alert: model.Alert{
Labels: model.LabelSet{
model.LabelName("instance"): model.LabelValue("two"),
},
Annotations: model.LabelSet{
model.LabelName(ruletypes.AnnotationBodyTemplate): model.LabelValue(bodyTpl),
},
},
}
alerts := []*types.Alert{a1, a2}
cfg := &config.EmailConfig{Headers: map[string]string{"Subject": "subj"}}
n := New(cfg, tmpl, promslog.NewNopLogger(), testTemplater(tmpl))
ctx := context.Background()
subject, htmlBody, err := n.prepareContent(ctx, alerts)
require.NoError(t, err)
require.Equal(t, "subj", subject)
require.Equal(t, "<div><p>line one</p>\n</div><div><p>line two</p>\n</div>", htmlBody)
})
t.Run("custom title template; default body HTML template", func(t *testing.T) {
tmpl, err := template.FromGlobs([]string{})
require.NoError(t, err)
tmpl.ExternalURL, _ = url.Parse("http://am")
firingAlert := &types.Alert{
Alert: model.Alert{
Labels: model.LabelSet{},
Annotations: model.LabelSet{
model.LabelName(ruletypes.AnnotationTitleTemplate): model.LabelValue("fixed from $alert.status"),
},
StartsAt: time.Now(),
EndsAt: time.Now().Add(time.Hour),
},
}
alerts := []*types.Alert{firingAlert}
cfg := &config.EmailConfig{
Headers: map[string]string{},
HTML: "Status: {{ .Status }}",
}
n := New(cfg, tmpl, promslog.NewNopLogger(), testTemplater(tmpl))
ctx := context.Background()
subject, htmlBody, err := n.prepareContent(ctx, alerts)
require.NoError(t, err)
require.Equal(t, "Status: firing", htmlBody)
require.Equal(t, "fixed from firing", subject)
})
t.Run("default template without HTML", func(t *testing.T) {
cfg := &config.EmailConfig{Headers: map[string]string{"Subject": "the email subject"}}
tmpl, err := template.FromGlobs([]string{})
require.NoError(t, err)
tmpl.ExternalURL, _ = url.Parse("http://am")
n := New(cfg, tmpl, promslog.NewNopLogger(), testTemplater(tmpl))
firingAlert := &types.Alert{
Alert: model.Alert{
Labels: model.LabelSet{},
StartsAt: time.Now(),
EndsAt: time.Now().Add(time.Hour),
},
}
alerts := []*types.Alert{firingAlert}
ctx := context.Background()
subject, htmlBody, err := n.prepareContent(ctx, alerts)
require.NoError(t, err)
require.Equal(t, "", htmlBody)
require.Equal(t, "the email subject", subject)
})
t.Run("custom title template; custom body template", func(t *testing.T) {
// Load the email.signoz.html layout into the notification template
// the same way the alertmanager server does via the templates config.
tmpl, err := template.FromGlobs([]string{"../../../../templates/alertmanager/*.gotmpl"})
require.NoError(t, err)
tmpl.ExternalURL, _ = url.Parse("http://am")
firingAlert := &types.Alert{
Alert: model.Alert{
Labels: model.LabelSet{
model.LabelName("instance"): model.LabelValue("two"),
},
Annotations: model.LabelSet{
model.LabelName(ruletypes.AnnotationTitleTemplate): model.LabelValue("fixed from $alert.status"),
model.LabelName(ruletypes.AnnotationBodyTemplate): model.LabelValue("line $labels.instance"),
},
StartsAt: time.Now(),
EndsAt: time.Now().Add(time.Hour),
},
}
alerts := []*types.Alert{firingAlert}
cfg := &config.EmailConfig{
Headers: map[string]string{"Subject": "subject"},
HTML: "Well, what are you?",
}
n := New(cfg, tmpl, promslog.NewNopLogger(), testTemplater(tmpl))
ctx := context.Background()
subject, htmlBody, err := n.prepareContent(ctx, alerts)
require.NoError(t, err)
require.Contains(t, htmlBody, "<!DOCTYPE html>")
require.Contains(t, htmlBody, "<p>line two</p>")
require.NotContains(t, htmlBody, "Well, what are you?")
require.Equal(t, subject, "fixed from firing")
require.NotContains(t, subject, "subject")
})
}
func ptrTo(b bool) *bool {
return &b
}

View File

@@ -15,7 +15,9 @@ import (
"slices"
"strings"
"github.com/SigNoz/signoz/pkg/alertmanager/alertmanagertemplate"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types/alertmanagertypes"
commoncfg "github.com/prometheus/common/config"
"github.com/prometheus/common/model"
@@ -44,6 +46,7 @@ type Notifier struct {
retrier *notify.Retrier
webhookURL *config.SecretURL
postJSONFunc func(ctx context.Context, client *http.Client, url string, body io.Reader) (*http.Response, error)
templater alertmanagertypes.Templater
}
// https://learn.microsoft.com/en-us/connectors/teams/?tabs=text1#adaptivecarditemschema
@@ -52,7 +55,7 @@ type Content struct {
Type string `json:"type"`
Version string `json:"version"`
Body []Body `json:"body"`
Msteams Msteams `json:"msteams,omitempty"`
Msteams Msteams `json:"msteams,omitzero"`
Actions []Action `json:"actions"`
}
@@ -94,7 +97,7 @@ type teamsMessage struct {
}
// New returns a new notifier that uses the Microsoft Teams Power Platform connector.
func New(c *config.MSTeamsV2Config, t *template.Template, titleLink string, l *slog.Logger, httpOpts ...commoncfg.HTTPClientOption) (*Notifier, error) {
func New(c *config.MSTeamsV2Config, t *template.Template, titleLink string, l *slog.Logger, templater alertmanagertypes.Templater, httpOpts ...commoncfg.HTTPClientOption) (*Notifier, error) {
client, err := notify.NewClientWithTracing(*c.HTTPConfig, Integration, httpOpts...)
if err != nil {
return nil, err
@@ -109,6 +112,7 @@ func New(c *config.MSTeamsV2Config, t *template.Template, titleLink string, l *s
retrier: &notify.Retrier{},
webhookURL: c.WebhookURL,
postJSONFunc: notify.PostJSON,
templater: templater,
}
return n, nil
@@ -128,25 +132,11 @@ func (n *Notifier) Notify(ctx context.Context, as ...*types.Alert) (bool, error)
return false, err
}
title := tmpl(n.conf.Title)
if err != nil {
return false, err
}
titleLink := tmpl(n.titleLink)
if err != nil {
return false, err
}
alerts := types.Alerts(as...)
color := colorGrey
switch alerts.Status() {
case model.AlertFiring:
color = colorRed
case model.AlertResolved:
color = colorGreen
}
var url string
if n.conf.WebhookURL != nil {
url = n.conf.WebhookURL.String()
@@ -158,6 +148,12 @@ func (n *Notifier) Notify(ctx context.Context, as ...*types.Alert) (bool, error)
url = strings.TrimSpace(string(content))
}
bodyBlocks, err := n.prepareContent(ctx, as)
if err != nil {
n.logger.ErrorContext(ctx, "failed to prepare notification content", errors.Attr(err))
return false, err
}
// A message as referenced in https://learn.microsoft.com/en-us/connectors/teams/?tabs=text1%2Cdotnet#request-body-schema
t := teamsMessage{
Type: "message",
@@ -169,17 +165,7 @@ func (n *Notifier) Notify(ctx context.Context, as ...*types.Alert) (bool, error)
Schema: "http://adaptivecards.io/schemas/adaptive-card.json",
Type: "AdaptiveCard",
Version: "1.2",
Body: []Body{
{
Type: "TextBlock",
Text: title,
Weight: "Bolder",
Size: "Medium",
Wrap: true,
Style: "heading",
Color: color,
},
},
Body: bodyBlocks,
Actions: []Action{
{
Type: "Action.OpenUrl",
@@ -195,20 +181,6 @@ func (n *Notifier) Notify(ctx context.Context, as ...*types.Alert) (bool, error)
},
}
// add labels and annotations to the body of all alerts
for _, alert := range as {
t.Attachments[0].Content.Body = append(t.Attachments[0].Content.Body, Body{
Type: "TextBlock",
Text: "Alerts",
Weight: "Bolder",
Size: "Medium",
Wrap: true,
Color: color,
})
t.Attachments[0].Content.Body = append(t.Attachments[0].Content.Body, n.createLabelsAndAnnotationsBody(alert)...)
}
var payload bytes.Buffer
if err = json.NewEncoder(&payload).Encode(t); err != nil {
return false, err
@@ -228,6 +200,75 @@ func (n *Notifier) Notify(ctx context.Context, as ...*types.Alert) (bool, error)
return shouldRetry, err
}
// prepareContent builds the Adaptive Card body blocks for the notification.
// The first block is always the title; the remainder depends on whether the
// alerts carried a custom body template.
func (n *Notifier) prepareContent(ctx context.Context, alerts []*types.Alert) ([]Body, error) {
customTitle, customBody := alertmanagertemplate.ExtractTemplatesFromAnnotations(alerts)
result, err := n.templater.Expand(ctx, alertmanagertypes.ExpandRequest{
TitleTemplate: customTitle,
BodyTemplate: customBody,
DefaultTitleTemplate: n.conf.Title,
DefaultBodyTemplate: n.conf.Text,
}, alerts)
if err != nil {
return nil, err
}
color := colorGrey
switch types.Alerts(alerts...).Status() {
case model.AlertFiring:
color = colorRed
case model.AlertResolved:
color = colorGreen
}
blocks := []Body{{
Type: "TextBlock",
Text: result.Title,
Weight: "Bolder",
Size: "Medium",
Wrap: true,
Style: "heading",
Color: color,
}}
if result.IsDefaultBody {
for _, alert := range alerts {
blocks = append(blocks, Body{
Type: "TextBlock",
Text: "Alerts",
Weight: "Bolder",
Size: "Medium",
Wrap: true,
Color: color,
})
blocks = append(blocks, n.createLabelsAndAnnotationsBody(alert)...)
}
return blocks, nil
}
// Custom body path: result.Body is positionally aligned with alerts;
// entries for alerts whose template rendered empty are kept as "" so we
// can skip them here without shifting the per-alert color index.
for i, body := range result.Body {
if body == "" || i >= len(alerts) {
continue
}
perAlertColor := colorRed
if alerts[i].Resolved() {
perAlertColor = colorGreen
}
blocks = append(blocks, Body{
Type: "TextBlock",
Text: body,
Wrap: true,
Color: perAlertColor,
})
}
return blocks, nil
}
func (*Notifier) createLabelsAndAnnotationsBody(alert *types.Alert) []Body {
bodies := []Body{}
bodies = append(bodies, Body{
@@ -258,7 +299,8 @@ func (*Notifier) createLabelsAndAnnotationsBody(alert *types.Alert) []Body {
annotationsFacts := []Fact{}
for k, v := range alert.Annotations {
if slices.Contains([]string{"summary", "related_logs", "related_traces"}, string(k)) {
if slices.Contains([]string{"summary", "related_logs", "related_traces"}, string(k)) ||
alertmanagertypes.IsPrivateAnnotation(string(k)) {
continue
}
annotationsFacts = append(annotationsFacts, Fact{Title: string(k), Value: string(v)})

View File

@@ -8,6 +8,7 @@ import (
"context"
"encoding/json"
"io"
"log/slog"
"net/http"
"net/http/httptest"
"net/url"
@@ -15,6 +16,9 @@ import (
"testing"
"time"
"github.com/SigNoz/signoz/pkg/alertmanager/alertmanagertemplate"
"github.com/SigNoz/signoz/pkg/types/alertmanagertypes"
"github.com/SigNoz/signoz/pkg/types/ruletypes"
commoncfg "github.com/prometheus/common/config"
"github.com/prometheus/common/model"
"github.com/prometheus/common/promslog"
@@ -23,21 +27,28 @@ import (
test "github.com/SigNoz/signoz/pkg/alertmanager/alertmanagernotify/alertmanagernotifytest"
"github.com/prometheus/alertmanager/config"
"github.com/prometheus/alertmanager/notify"
"github.com/prometheus/alertmanager/template"
"github.com/prometheus/alertmanager/types"
)
func newTestTemplater(tmpl *template.Template) alertmanagertypes.Templater {
return alertmanagertemplate.New(tmpl, slog.New(slog.DiscardHandler))
}
// This is a test URL that has been modified to not be valid.
var testWebhookURL, _ = url.Parse("https://example.westeurope.logic.azure.com:443/workflows/xxx/triggers/manual/paths/invoke?api-version=2016-06-01&sp=%2Ftriggers%2Fmanual%2Frun&sv=1.0&sig=xxx")
func TestMSTeamsV2Retry(t *testing.T) {
tmpl := test.CreateTmpl(t)
notifier, err := New(
&config.MSTeamsV2Config{
WebhookURL: &config.SecretURL{URL: testWebhookURL},
HTTPConfig: &commoncfg.HTTPClientConfig{},
},
test.CreateTmpl(t),
tmpl,
`{{ template "msteamsv2.default.titleLink" . }}`,
promslog.NewNopLogger(),
newTestTemplater(tmpl),
)
require.NoError(t, err)
@@ -64,14 +75,16 @@ func TestNotifier_Notify_WithReason(t *testing.T) {
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
tmpl := test.CreateTmpl(t)
notifier, err := New(
&config.MSTeamsV2Config{
WebhookURL: &config.SecretURL{URL: testWebhookURL},
HTTPConfig: &commoncfg.HTTPClientConfig{},
},
test.CreateTmpl(t),
tmpl,
`{{ template "msteamsv2.default.titleLink" . }}`,
promslog.NewNopLogger(),
newTestTemplater(tmpl),
)
require.NoError(t, err)
@@ -153,7 +166,8 @@ func TestMSTeamsV2Templating(t *testing.T) {
t.Run(tc.title, func(t *testing.T) {
tc.cfg.WebhookURL = &config.SecretURL{URL: u}
tc.cfg.HTTPConfig = &commoncfg.HTTPClientConfig{}
pd, err := New(tc.cfg, test.CreateTmpl(t), tc.titleLink, promslog.NewNopLogger())
tmpl := test.CreateTmpl(t)
pd, err := New(tc.cfg, tmpl, tc.titleLink, promslog.NewNopLogger(), newTestTemplater(tmpl))
require.NoError(t, err)
ctx := context.Background()
@@ -186,20 +200,124 @@ func TestMSTeamsV2RedactedURL(t *testing.T) {
defer fn()
secret := "secret"
tmpl := test.CreateTmpl(t)
notifier, err := New(
&config.MSTeamsV2Config{
WebhookURL: &config.SecretURL{URL: u},
HTTPConfig: &commoncfg.HTTPClientConfig{},
},
test.CreateTmpl(t),
tmpl,
`{{ template "msteamsv2.default.titleLink" . }}`,
promslog.NewNopLogger(),
newTestTemplater(tmpl),
)
require.NoError(t, err)
test.AssertNotifyLeaksNoSecret(ctx, t, notifier, secret)
}
func TestPrepareContent(t *testing.T) {
t.Run("default template - firing alerts", func(t *testing.T) {
tmpl := test.CreateTmpl(t)
notifier, err := New(
&config.MSTeamsV2Config{
WebhookURL: &config.SecretURL{URL: testWebhookURL},
HTTPConfig: &commoncfg.HTTPClientConfig{},
Title: "Alertname: {{ .CommonLabels.alertname }}",
},
tmpl,
`{{ template "msteamsv2.default.titleLink" . }}`,
promslog.NewNopLogger(),
newTestTemplater(tmpl),
)
require.NoError(t, err)
ctx := context.Background()
ctx = notify.WithGroupKey(ctx, "1")
alerts := []*types.Alert{
{
Alert: model.Alert{
Labels: model.LabelSet{"alertname": "test"},
// Custom body template
Annotations: model.LabelSet{
ruletypes.AnnotationBodyTemplate: "Firing alert: $alertname",
},
StartsAt: time.Now(),
EndsAt: time.Now().Add(time.Hour),
},
},
}
blocks, err := notifier.prepareContent(ctx, alerts)
require.NoError(t, err)
require.NotEmpty(t, blocks)
// First block should be the title with color (firing = red)
require.Equal(t, "Bolder", blocks[0].Weight)
require.Equal(t, colorRed, blocks[0].Color)
// verify title text
require.Equal(t, "Alertname: test", blocks[0].Text)
// verify body text
require.Equal(t, "Firing alert: test", blocks[1].Text)
})
t.Run("custom template - per-alert color", func(t *testing.T) {
tmpl := test.CreateTmpl(t)
notifier, err := New(
&config.MSTeamsV2Config{
WebhookURL: &config.SecretURL{URL: testWebhookURL},
HTTPConfig: &commoncfg.HTTPClientConfig{},
},
tmpl,
`{{ template "msteamsv2.default.titleLink" . }}`,
promslog.NewNopLogger(),
newTestTemplater(tmpl),
)
require.NoError(t, err)
ctx := context.Background()
ctx = notify.WithGroupKey(ctx, "1")
alerts := []*types.Alert{
{
Alert: model.Alert{
Labels: model.LabelSet{"alertname": "test1"},
Annotations: model.LabelSet{
"summary": "test",
ruletypes.AnnotationTitleTemplate: "Custom Title",
ruletypes.AnnotationBodyTemplate: "custom body $alertname",
},
StartsAt: time.Now(),
EndsAt: time.Now().Add(time.Hour),
},
},
{
Alert: model.Alert{
Labels: model.LabelSet{"alertname": "test2"},
Annotations: model.LabelSet{
"summary": "test",
ruletypes.AnnotationTitleTemplate: "Custom Title",
ruletypes.AnnotationBodyTemplate: "custom body $alertname",
},
StartsAt: time.Now().Add(-time.Hour),
EndsAt: time.Now().Add(-time.Minute),
},
},
}
blocks, err := notifier.prepareContent(ctx, alerts)
require.NoError(t, err)
require.NotEmpty(t, blocks)
// total 3 blocks: title and 2 body blocks
require.True(t, len(blocks) == 3)
// First block: title color is overall color of the alerts
require.Equal(t, colorRed, blocks[0].Color)
// verify title text
require.Equal(t, "Custom Title", blocks[0].Text)
// Body blocks should have per-alert color
require.Equal(t, colorRed, blocks[1].Color) // firing
require.Equal(t, colorGreen, blocks[2].Color) // resolved
})
}
func TestMSTeamsV2ReadingURLFromFile(t *testing.T) {
ctx, u, fn := test.GetContextWithCancelingURL()
defer fn()
@@ -209,14 +327,16 @@ func TestMSTeamsV2ReadingURLFromFile(t *testing.T) {
_, err = f.WriteString(u.String() + "\n")
require.NoError(t, err, "writing to temp file failed")
tmpl := test.CreateTmpl(t)
notifier, err := New(
&config.MSTeamsV2Config{
WebhookURLFile: f.Name(),
HTTPConfig: &commoncfg.HTTPClientConfig{},
},
test.CreateTmpl(t),
tmpl,
`{{ template "msteamsv2.default.titleLink" . }}`,
promslog.NewNopLogger(),
newTestTemplater(tmpl),
)
require.NoError(t, err)

View File

@@ -15,7 +15,10 @@ import (
"os"
"strings"
"github.com/SigNoz/signoz/pkg/alertmanager/alertmanagertemplate"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/templating/markdownrenderer"
"github.com/SigNoz/signoz/pkg/types/alertmanagertypes"
commoncfg "github.com/prometheus/common/config"
"github.com/prometheus/common/model"
@@ -34,25 +37,27 @@ const maxMessageLenRunes = 130
// Notifier implements a Notifier for OpsGenie notifications.
type Notifier struct {
conf *config.OpsGenieConfig
tmpl *template.Template
logger *slog.Logger
client *http.Client
retrier *notify.Retrier
conf *config.OpsGenieConfig
tmpl *template.Template
logger *slog.Logger
client *http.Client
retrier *notify.Retrier
templater alertmanagertypes.Templater
}
// New returns a new OpsGenie notifier.
func New(c *config.OpsGenieConfig, t *template.Template, l *slog.Logger, httpOpts ...commoncfg.HTTPClientOption) (*Notifier, error) {
func New(c *config.OpsGenieConfig, t *template.Template, l *slog.Logger, templater alertmanagertypes.Templater, httpOpts ...commoncfg.HTTPClientOption) (*Notifier, error) {
client, err := notify.NewClientWithTracing(*c.HTTPConfig, Integration, httpOpts...)
if err != nil {
return nil, err
}
return &Notifier{
conf: c,
tmpl: t,
logger: l,
client: client,
retrier: &notify.Retrier{RetryCodes: []int{http.StatusTooManyRequests}},
conf: c,
tmpl: t,
logger: l,
client: client,
retrier: &notify.Retrier{RetryCodes: []int{http.StatusTooManyRequests}},
templater: templater,
}, nil
}
@@ -123,6 +128,55 @@ func safeSplit(s, sep string) []string {
return b
}
// prepareContent expands alert templates and returns the OpsGenie-ready title
// (truncated to the 130-rune limit) and HTML description. Custom bodies are
// rendered to HTML and stitched together with <hr> dividers; default bodies
// are joined with newlines (OpsGenie's legacy plain-text description).
func (n *Notifier) prepareContent(ctx context.Context, alerts []*types.Alert) (string, string, error) {
customTitle, customBody := alertmanagertemplate.ExtractTemplatesFromAnnotations(alerts)
result, err := n.templater.Expand(ctx, alertmanagertypes.ExpandRequest{
TitleTemplate: customTitle,
BodyTemplate: customBody,
DefaultTitleTemplate: n.conf.Message,
DefaultBodyTemplate: n.conf.Description,
}, alerts)
if err != nil {
return "", "", err
}
var description string
if result.IsDefaultBody {
description = strings.Join(result.Body, "\n")
} else {
var b strings.Builder
first := true
for _, part := range result.Body {
if part == "" {
continue
}
rendered, renderErr := markdownrenderer.RenderHTML(part)
if renderErr != nil {
return "", "", renderErr
}
if !first {
b.WriteString("<hr>")
}
b.WriteString("<div>")
b.WriteString(rendered)
b.WriteString("</div>")
first = false
}
description = b.String()
}
title, truncated := notify.TruncateInRunes(result.Title, maxMessageLenRunes)
if truncated {
n.logger.WarnContext(ctx, "Truncated message", slog.Int("max_runes", maxMessageLenRunes))
}
return title, description, nil
}
// Create requests for a list of alerts.
func (n *Notifier) createRequests(ctx context.Context, as ...*types.Alert) ([]*http.Request, bool, error) {
key, err := notify.ExtractGroupKey(ctx)
@@ -168,9 +222,10 @@ func (n *Notifier) createRequests(ctx context.Context, as ...*types.Alert) ([]*h
}
requests = append(requests, req.WithContext(ctx))
default:
message, truncated := notify.TruncateInRunes(tmpl(n.conf.Message), maxMessageLenRunes)
if truncated {
logger.WarnContext(ctx, "Truncated message", slog.Any("alert", key), slog.Int("max_runes", maxMessageLenRunes))
message, description, err := n.prepareContent(ctx, as)
if err != nil {
n.logger.ErrorContext(ctx, "failed to prepare notification content", errors.Attr(err))
return nil, false, err
}
createEndpointURL := n.conf.APIURL.Copy()
@@ -209,7 +264,7 @@ func (n *Notifier) createRequests(ctx context.Context, as ...*types.Alert) ([]*h
msg := &opsGenieCreateMessage{
Alias: alias,
Message: message,
Description: tmpl(n.conf.Description),
Description: description,
Details: details,
Source: tmpl(n.conf.Source),
Responders: responders,

View File

@@ -8,12 +8,16 @@ import (
"context"
"fmt"
"io"
"log/slog"
"net/http"
"net/url"
"os"
"testing"
"time"
"github.com/SigNoz/signoz/pkg/alertmanager/alertmanagertemplate"
"github.com/SigNoz/signoz/pkg/types/alertmanagertypes"
"github.com/SigNoz/signoz/pkg/types/ruletypes"
commoncfg "github.com/prometheus/common/config"
"github.com/prometheus/common/model"
"github.com/prometheus/common/promslog"
@@ -22,16 +26,23 @@ import (
"github.com/prometheus/alertmanager/config"
"github.com/prometheus/alertmanager/notify"
"github.com/prometheus/alertmanager/notify/test"
"github.com/prometheus/alertmanager/template"
"github.com/prometheus/alertmanager/types"
)
func newTestTemplater(tmpl *template.Template) alertmanagertypes.Templater {
return alertmanagertemplate.New(tmpl, slog.New(slog.DiscardHandler))
}
func TestOpsGenieRetry(t *testing.T) {
tmpl := test.CreateTmpl(t)
notifier, err := New(
&config.OpsGenieConfig{
HTTPConfig: &commoncfg.HTTPClientConfig{},
},
test.CreateTmpl(t),
tmpl,
promslog.NewNopLogger(),
newTestTemplater(tmpl),
)
require.NoError(t, err)
@@ -47,14 +58,16 @@ func TestOpsGenieRedactedURL(t *testing.T) {
defer fn()
key := "key"
tmpl := test.CreateTmpl(t)
notifier, err := New(
&config.OpsGenieConfig{
APIURL: &config.URL{URL: u},
APIKey: config.Secret(key),
HTTPConfig: &commoncfg.HTTPClientConfig{},
},
test.CreateTmpl(t),
tmpl,
promslog.NewNopLogger(),
newTestTemplater(tmpl),
)
require.NoError(t, err)
@@ -72,14 +85,16 @@ func TestGettingOpsGegineApikeyFromFile(t *testing.T) {
_, err = f.WriteString(key)
require.NoError(t, err, "writing to temp file failed")
tmpl := test.CreateTmpl(t)
notifier, err := New(
&config.OpsGenieConfig{
APIURL: &config.URL{URL: u},
APIKeyFile: f.Name(),
HTTPConfig: &commoncfg.HTTPClientConfig{},
},
test.CreateTmpl(t),
tmpl,
promslog.NewNopLogger(),
newTestTemplater(tmpl),
)
require.NoError(t, err)
@@ -202,7 +217,7 @@ func TestOpsGenie(t *testing.T) {
},
} {
t.Run(tc.title, func(t *testing.T) {
notifier, err := New(tc.cfg, tmpl, logger)
notifier, err := New(tc.cfg, tmpl, logger, newTestTemplater(tmpl))
require.NoError(t, err)
ctx := context.Background()
@@ -278,7 +293,7 @@ func TestOpsGenieWithUpdate(t *testing.T) {
APIURL: &config.URL{URL: u},
HTTPConfig: &commoncfg.HTTPClientConfig{},
}
notifierWithUpdate, err := New(&opsGenieConfigWithUpdate, tmpl, promslog.NewNopLogger())
notifierWithUpdate, err := New(&opsGenieConfigWithUpdate, tmpl, promslog.NewNopLogger(), newTestTemplater(tmpl))
alert := &types.Alert{
Alert: model.Alert{
StartsAt: time.Now(),
@@ -321,7 +336,7 @@ func TestOpsGenieApiKeyFile(t *testing.T) {
APIURL: &config.URL{URL: u},
HTTPConfig: &commoncfg.HTTPClientConfig{},
}
notifierWithUpdate, err := New(&opsGenieConfigWithUpdate, tmpl, promslog.NewNopLogger())
notifierWithUpdate, err := New(&opsGenieConfigWithUpdate, tmpl, promslog.NewNopLogger(), newTestTemplater(tmpl))
require.NoError(t, err)
requests, _, err := notifierWithUpdate.createRequests(ctx)
@@ -329,6 +344,99 @@ func TestOpsGenieApiKeyFile(t *testing.T) {
require.Equal(t, "GenieKey my_secret_api_key", requests[0].Header.Get("Authorization"))
}
func TestPrepareContent(t *testing.T) {
t.Run("default template", func(t *testing.T) {
tmpl := test.CreateTmpl(t)
logger := promslog.NewNopLogger()
notifier := &Notifier{
conf: &config.OpsGenieConfig{
Message: `{{ .CommonLabels.Message }}`,
Description: `{{ .CommonLabels.Description }}`,
},
tmpl: tmpl,
logger: logger,
templater: newTestTemplater(tmpl),
}
ctx := context.Background()
ctx = notify.WithGroupKey(ctx, "1")
alert := &types.Alert{
Alert: model.Alert{
Labels: model.LabelSet{
"Message": "Firing alert: test",
"Description": "Check runbook for more details",
},
StartsAt: time.Now(),
EndsAt: time.Now().Add(time.Hour),
},
}
alerts := []*types.Alert{alert}
title, desc, prepErr := notifier.prepareContent(ctx, alerts)
require.NoError(t, prepErr)
require.Equal(t, "Firing alert: test", title)
require.Equal(t, "Check runbook for more details", desc)
})
t.Run("custom template", func(t *testing.T) {
tmpl := test.CreateTmpl(t)
logger := promslog.NewNopLogger()
notifier := &Notifier{
conf: &config.OpsGenieConfig{
Message: `{{ .CommonLabels.Message }}`,
Description: `{{ .CommonLabels.Description }}`,
},
tmpl: tmpl,
logger: logger,
templater: newTestTemplater(tmpl),
}
ctx := context.Background()
ctx = notify.WithGroupKey(ctx, "1")
alert1 := &types.Alert{
Alert: model.Alert{
Labels: model.LabelSet{
"service": "payment",
"namespace": "potter-the-harry",
},
Annotations: model.LabelSet{
ruletypes.AnnotationTitleTemplate: "High request throughput for $service",
ruletypes.AnnotationBodyTemplate: "Alert firing in NS: $labels.namespace",
},
StartsAt: time.Now(),
EndsAt: time.Now().Add(time.Hour),
},
}
alert2 := &types.Alert{
Alert: model.Alert{
Labels: model.LabelSet{
"service": "payment",
"namespace": "smart-the-rat",
},
Annotations: model.LabelSet{
ruletypes.AnnotationTitleTemplate: "High request throughput for $service",
ruletypes.AnnotationBodyTemplate: "Alert firing in NS: $labels.namespace",
},
StartsAt: time.Now(),
EndsAt: time.Now().Add(time.Hour),
},
}
alerts := []*types.Alert{alert1, alert2}
title, desc, err := notifier.prepareContent(ctx, alerts)
require.NoError(t, err)
require.Equal(t, "High request throughput for payment", title)
// Each alert body wrapped in <div>, separated by <hr>
require.Equal(t, "<div><p>Alert firing in NS: potter-the-harry</p>\n</div><hr><div><p>Alert firing in NS: smart-the-rat</p>\n</div>", desc)
})
}
func readBody(t *testing.T, r *http.Request) string {
t.Helper()
body, err := io.ReadAll(r.Body)

View File

@@ -15,7 +15,9 @@ import (
"os"
"strings"
"github.com/SigNoz/signoz/pkg/alertmanager/alertmanagertemplate"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types/alertmanagertypes"
"github.com/alecthomas/units"
commoncfg "github.com/prometheus/common/config"
"github.com/prometheus/common/model"
@@ -40,21 +42,22 @@ const (
// Notifier implements a Notifier for PagerDuty notifications.
type Notifier struct {
conf *config.PagerdutyConfig
tmpl *template.Template
logger *slog.Logger
apiV1 string // for tests.
client *http.Client
retrier *notify.Retrier
conf *config.PagerdutyConfig
tmpl *template.Template
logger *slog.Logger
apiV1 string // for tests.
client *http.Client
retrier *notify.Retrier
templater alertmanagertypes.Templater
}
// New returns a new PagerDuty notifier.
func New(c *config.PagerdutyConfig, t *template.Template, l *slog.Logger, httpOpts ...commoncfg.HTTPClientOption) (*Notifier, error) {
func New(c *config.PagerdutyConfig, t *template.Template, l *slog.Logger, templater alertmanagertypes.Templater, httpOpts ...commoncfg.HTTPClientOption) (*Notifier, error) {
client, err := notify.NewClientWithTracing(*c.HTTPConfig, Integration, httpOpts...)
if err != nil {
return nil, err
}
n := &Notifier{conf: c, tmpl: t, logger: l, client: client}
n := &Notifier{conf: c, tmpl: t, logger: l, client: client, templater: templater}
if c.ServiceKey != "" || c.ServiceKeyFile != "" {
n.apiV1 = "https://events.pagerduty.com/generic/2010-04-15/create_event.json"
// Retrying can solve the issue on 403 (rate limiting) and 5xx response codes.
@@ -143,11 +146,12 @@ func (n *Notifier) notifyV1(
key notify.Key,
data *template.Data,
details map[string]any,
title string,
) (bool, error) {
var tmplErr error
tmpl := notify.TmplText(n.tmpl, data, &tmplErr)
description, truncated := notify.TruncateInRunes(tmpl(n.conf.Description), maxV1DescriptionLenRunes)
description, truncated := notify.TruncateInRunes(title, maxV1DescriptionLenRunes)
if truncated {
n.logger.WarnContext(ctx, "Truncated description", slog.Any("key", key), slog.Int("max_runes", maxV1DescriptionLenRunes))
}
@@ -203,6 +207,7 @@ func (n *Notifier) notifyV2(
key notify.Key,
data *template.Data,
details map[string]any,
title string,
) (bool, error) {
var tmplErr error
tmpl := notify.TmplText(n.tmpl, data, &tmplErr)
@@ -211,7 +216,7 @@ func (n *Notifier) notifyV2(
n.conf.Severity = "error"
}
summary, truncated := notify.TruncateInRunes(tmpl(n.conf.Description), maxV2SummaryLenRunes)
summary, truncated := notify.TruncateInRunes(title, maxV2SummaryLenRunes)
if truncated {
n.logger.WarnContext(ctx, "Truncated summary", slog.Any("key", key), slog.Int("max_runes", maxV2SummaryLenRunes))
}
@@ -294,6 +299,22 @@ func (n *Notifier) notifyV2(
return retry, err
}
// prepareTitle expands the notification title. PagerDuty has no body surface
// we care about — the description/summary field is what users see as the
// incident headline, so we feed the configured Description as the default
// title template and ignore any custom body_template entirely.
func (n *Notifier) prepareTitle(ctx context.Context, alerts []*types.Alert) (string, error) {
customTitle, _ := alertmanagertemplate.ExtractTemplatesFromAnnotations(alerts)
result, err := n.templater.Expand(ctx, alertmanagertypes.ExpandRequest{
TitleTemplate: customTitle,
DefaultTitleTemplate: n.conf.Description,
}, alerts)
if err != nil {
return "", err
}
return result.Title, nil
}
// Notify implements the Notifier interface.
func (n *Notifier) Notify(ctx context.Context, as ...*types.Alert) (bool, error) {
key, err := notify.ExtractGroupKey(ctx)
@@ -302,6 +323,12 @@ func (n *Notifier) Notify(ctx context.Context, as ...*types.Alert) (bool, error)
}
logger := n.logger.With(slog.Any("group_key", key))
title, err := n.prepareTitle(ctx, as)
if err != nil {
n.logger.ErrorContext(ctx, "failed to prepare notification content", errors.Attr(err))
return false, err
}
var (
alerts = types.Alerts(as...)
data = notify.GetTemplateData(ctx, n.tmpl, as, logger)
@@ -329,7 +356,7 @@ func (n *Notifier) Notify(ctx context.Context, as ...*types.Alert) (bool, error)
if n.apiV1 != "" {
nf = n.notifyV1
}
retry, err := nf(ctx, eventType, key, data, details)
retry, err := nf(ctx, eventType, key, data, details, title)
if err != nil {
if ctx.Err() != nil {
err = errors.WrapInternalf(err, errors.CodeInternal, "failed to notify PagerDuty: %v", context.Cause(ctx))

View File

@@ -9,6 +9,7 @@ import (
"context"
"encoding/json"
"io"
"log/slog"
"net/http"
"net/http/httptest"
"net/url"
@@ -17,7 +18,10 @@ import (
"testing"
"time"
"github.com/SigNoz/signoz/pkg/alertmanager/alertmanagertemplate"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types/alertmanagertypes"
"github.com/SigNoz/signoz/pkg/types/ruletypes"
commoncfg "github.com/prometheus/common/config"
"github.com/prometheus/common/model"
"github.com/prometheus/common/promslog"
@@ -30,14 +34,20 @@ import (
"github.com/prometheus/alertmanager/types"
)
func newTestTemplater(tmpl *template.Template) alertmanagertypes.Templater {
return alertmanagertemplate.New(tmpl, slog.New(slog.DiscardHandler))
}
func TestPagerDutyRetryV1(t *testing.T) {
tmpl := test.CreateTmpl(t)
notifier, err := New(
&config.PagerdutyConfig{
ServiceKey: config.Secret("01234567890123456789012345678901"),
HTTPConfig: &commoncfg.HTTPClientConfig{},
},
test.CreateTmpl(t),
tmpl,
promslog.NewNopLogger(),
newTestTemplater(tmpl),
)
require.NoError(t, err)
@@ -49,13 +59,15 @@ func TestPagerDutyRetryV1(t *testing.T) {
}
func TestPagerDutyRetryV2(t *testing.T) {
tmpl := test.CreateTmpl(t)
notifier, err := New(
&config.PagerdutyConfig{
RoutingKey: config.Secret("01234567890123456789012345678901"),
HTTPConfig: &commoncfg.HTTPClientConfig{},
},
test.CreateTmpl(t),
tmpl,
promslog.NewNopLogger(),
newTestTemplater(tmpl),
)
require.NoError(t, err)
@@ -71,13 +83,15 @@ func TestPagerDutyRedactedURLV1(t *testing.T) {
defer fn()
key := "01234567890123456789012345678901"
tmpl := test.CreateTmpl(t)
notifier, err := New(
&config.PagerdutyConfig{
ServiceKey: config.Secret(key),
HTTPConfig: &commoncfg.HTTPClientConfig{},
},
test.CreateTmpl(t),
tmpl,
promslog.NewNopLogger(),
newTestTemplater(tmpl),
)
require.NoError(t, err)
notifier.apiV1 = u.String()
@@ -90,14 +104,16 @@ func TestPagerDutyRedactedURLV2(t *testing.T) {
defer fn()
key := "01234567890123456789012345678901"
tmpl := test.CreateTmpl(t)
notifier, err := New(
&config.PagerdutyConfig{
URL: &config.URL{URL: u},
RoutingKey: config.Secret(key),
HTTPConfig: &commoncfg.HTTPClientConfig{},
},
test.CreateTmpl(t),
tmpl,
promslog.NewNopLogger(),
newTestTemplater(tmpl),
)
require.NoError(t, err)
@@ -114,13 +130,15 @@ func TestPagerDutyV1ServiceKeyFromFile(t *testing.T) {
ctx, u, fn := test.GetContextWithCancelingURL()
defer fn()
tmpl := test.CreateTmpl(t)
notifier, err := New(
&config.PagerdutyConfig{
ServiceKeyFile: f.Name(),
HTTPConfig: &commoncfg.HTTPClientConfig{},
},
test.CreateTmpl(t),
tmpl,
promslog.NewNopLogger(),
newTestTemplater(tmpl),
)
require.NoError(t, err)
notifier.apiV1 = u.String()
@@ -138,14 +156,16 @@ func TestPagerDutyV2RoutingKeyFromFile(t *testing.T) {
ctx, u, fn := test.GetContextWithCancelingURL()
defer fn()
tmpl := test.CreateTmpl(t)
notifier, err := New(
&config.PagerdutyConfig{
URL: &config.URL{URL: u},
RoutingKeyFile: f.Name(),
HTTPConfig: &commoncfg.HTTPClientConfig{},
},
test.CreateTmpl(t),
tmpl,
promslog.NewNopLogger(),
newTestTemplater(tmpl),
)
require.NoError(t, err)
@@ -302,7 +322,8 @@ func TestPagerDutyTemplating(t *testing.T) {
t.Run(tc.title, func(t *testing.T) {
tc.cfg.URL = &config.URL{URL: u}
tc.cfg.HTTPConfig = &commoncfg.HTTPClientConfig{}
pd, err := New(tc.cfg, test.CreateTmpl(t), promslog.NewNopLogger())
tmpl := test.CreateTmpl(t)
pd, err := New(tc.cfg, tmpl, promslog.NewNopLogger(), newTestTemplater(tmpl))
require.NoError(t, err)
if pd.apiV1 != "" {
pd.apiV1 = u.String()
@@ -392,13 +413,15 @@ func TestEventSizeEnforcement(t *testing.T) {
Details: bigDetailsV1,
}
tmpl := test.CreateTmpl(t)
notifierV1, err := New(
&config.PagerdutyConfig{
ServiceKey: config.Secret("01234567890123456789012345678901"),
HTTPConfig: &commoncfg.HTTPClientConfig{},
},
test.CreateTmpl(t),
tmpl,
promslog.NewNopLogger(),
newTestTemplater(tmpl),
)
require.NoError(t, err)
@@ -420,8 +443,9 @@ func TestEventSizeEnforcement(t *testing.T) {
RoutingKey: config.Secret("01234567890123456789012345678901"),
HTTPConfig: &commoncfg.HTTPClientConfig{},
},
test.CreateTmpl(t),
tmpl,
promslog.NewNopLogger(),
newTestTemplater(tmpl),
)
require.NoError(t, err)
@@ -536,7 +560,8 @@ func TestPagerDutyEmptySrcHref(t *testing.T) {
Links: links,
}
pagerDuty, err := New(&pagerDutyConfig, test.CreateTmpl(t), promslog.NewNopLogger())
pdTmpl := test.CreateTmpl(t)
pagerDuty, err := New(&pagerDutyConfig, pdTmpl, promslog.NewNopLogger(), newTestTemplater(pdTmpl))
require.NoError(t, err)
ctx := context.Background()
@@ -603,7 +628,8 @@ func TestPagerDutyTimeout(t *testing.T) {
Timeout: tt.timeout,
}
pd, err := New(&cfg, test.CreateTmpl(t), promslog.NewNopLogger())
tmpl := test.CreateTmpl(t)
pd, err := New(&cfg, tmpl, promslog.NewNopLogger(), newTestTemplater(tmpl))
require.NoError(t, err)
ctx := context.Background()
@@ -881,3 +907,79 @@ func TestRenderDetails(t *testing.T) {
})
}
}
func TestPrepareContent(t *testing.T) {
prepareContext := func() context.Context {
ctx := context.Background()
ctx = notify.WithGroupKey(ctx, "1")
ctx = notify.WithReceiverName(ctx, "test-receiver")
ctx = notify.WithGroupLabels(ctx, model.LabelSet{"alertname": "HighCPU for Payment service"})
return ctx
}
t.Run("default template uses go text template config for title", func(t *testing.T) {
tmpl := test.CreateTmpl(t)
notifier, err := New(
&config.PagerdutyConfig{
RoutingKey: config.Secret("01234567890123456789012345678901"),
HTTPConfig: &commoncfg.HTTPClientConfig{},
Description: `{{ .CommonLabels.alertname }} ({{ .Status | toUpper }})`,
},
tmpl,
promslog.NewNopLogger(),
newTestTemplater(tmpl),
)
require.NoError(t, err)
ctx := prepareContext()
alerts := []*types.Alert{
{
Alert: model.Alert{
Labels: model.LabelSet{"alertname": "HighCPU for Payment service"},
StartsAt: time.Now(),
EndsAt: time.Now().Add(time.Hour),
},
},
}
title, err := notifier.prepareTitle(ctx, alerts)
require.NoError(t, err)
require.Equal(t, "HighCPU for Payment service (FIRING)", title)
})
t.Run("custom template uses $variable annotation for title", func(t *testing.T) {
tmpl := test.CreateTmpl(t)
notifier, err := New(
&config.PagerdutyConfig{
RoutingKey: config.Secret("01234567890123456789012345678901"),
HTTPConfig: &commoncfg.HTTPClientConfig{},
},
tmpl,
promslog.NewNopLogger(),
newTestTemplater(tmpl),
)
require.NoError(t, err)
ctx := prepareContext()
alerts := []*types.Alert{
{
Alert: model.Alert{
Labels: model.LabelSet{
"alertname": "HighCPU",
"service": "api-server",
},
Annotations: model.LabelSet{
ruletypes.AnnotationTitleTemplate: "$rule.name on $service is in $alert.status state",
},
StartsAt: time.Now().Add(-time.Hour),
EndsAt: time.Now(),
},
},
}
title, err := notifier.prepareTitle(ctx, alerts)
require.NoError(t, err)
require.Equal(t, "HighCPU on api-server is in resolved state", title)
})
}

View File

@@ -26,7 +26,7 @@ var customNotifierIntegrations = []string{
msteamsv2.Integration,
}
func NewReceiverIntegrations(nc alertmanagertypes.Receiver, tmpl *template.Template, logger *slog.Logger) ([]notify.Integration, error) {
func NewReceiverIntegrations(nc alertmanagertypes.Receiver, tmpl *template.Template, logger *slog.Logger, templater alertmanagertypes.Templater) ([]notify.Integration, error) {
upstreamIntegrations, err := receiver.BuildReceiverIntegrations(nc, tmpl, logger)
if err != nil {
return nil, err
@@ -53,23 +53,25 @@ func NewReceiverIntegrations(nc alertmanagertypes.Receiver, tmpl *template.Templ
}
for i, c := range nc.WebhookConfigs {
add(webhook.Integration, i, c, func(l *slog.Logger) (notify.Notifier, error) { return webhook.New(c, tmpl, l) })
add(webhook.Integration, i, c, func(l *slog.Logger) (notify.Notifier, error) { return webhook.New(c, tmpl, l, templater) })
}
for i, c := range nc.EmailConfigs {
add(email.Integration, i, c, func(l *slog.Logger) (notify.Notifier, error) { return email.New(c, tmpl, l), nil })
add(email.Integration, i, c, func(l *slog.Logger) (notify.Notifier, error) {
return email.New(c, tmpl, l, templater), nil
})
}
for i, c := range nc.PagerdutyConfigs {
add(pagerduty.Integration, i, c, func(l *slog.Logger) (notify.Notifier, error) { return pagerduty.New(c, tmpl, l) })
add(pagerduty.Integration, i, c, func(l *slog.Logger) (notify.Notifier, error) { return pagerduty.New(c, tmpl, l, templater) })
}
for i, c := range nc.OpsGenieConfigs {
add(opsgenie.Integration, i, c, func(l *slog.Logger) (notify.Notifier, error) { return opsgenie.New(c, tmpl, l) })
add(opsgenie.Integration, i, c, func(l *slog.Logger) (notify.Notifier, error) { return opsgenie.New(c, tmpl, l, templater) })
}
for i, c := range nc.SlackConfigs {
add(slack.Integration, i, c, func(l *slog.Logger) (notify.Notifier, error) { return slack.New(c, tmpl, l) })
add(slack.Integration, i, c, func(l *slog.Logger) (notify.Notifier, error) { return slack.New(c, tmpl, l, templater) })
}
for i, c := range nc.MSTeamsV2Configs {
add(msteamsv2.Integration, i, c, func(l *slog.Logger) (notify.Notifier, error) {
return msteamsv2.New(c, tmpl, `{{ template "msteamsv2.default.titleLink" . }}`, l)
return msteamsv2.New(c, tmpl, `{{ template "msteamsv2.default.titleLink" . }}`, l, templater)
})
}

View File

@@ -14,7 +14,11 @@ import (
"os"
"strings"
"github.com/SigNoz/signoz/pkg/alertmanager/alertmanagertemplate"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/templating/markdownrenderer"
"github.com/SigNoz/signoz/pkg/types/alertmanagertypes"
"github.com/SigNoz/signoz/pkg/types/ruletypes"
commoncfg "github.com/prometheus/common/config"
"github.com/prometheus/alertmanager/config"
@@ -25,6 +29,8 @@ import (
const (
Integration = "slack"
colorRed = "#FF0000"
colorGreen = "#00FF00"
)
// https://api.slack.com/reference/messaging/attachments#legacy_fields - 1024, no units given, assuming runes or characters.
@@ -32,17 +38,18 @@ const maxTitleLenRunes = 1024
// Notifier implements a Notifier for Slack notifications.
type Notifier struct {
conf *config.SlackConfig
tmpl *template.Template
logger *slog.Logger
client *http.Client
retrier *notify.Retrier
conf *config.SlackConfig
tmpl *template.Template
logger *slog.Logger
client *http.Client
retrier *notify.Retrier
templater alertmanagertypes.Templater
postJSONFunc func(ctx context.Context, client *http.Client, url string, body io.Reader) (*http.Response, error)
}
// New returns a new Slack notification handler.
func New(c *config.SlackConfig, t *template.Template, l *slog.Logger, httpOpts ...commoncfg.HTTPClientOption) (*Notifier, error) {
func New(c *config.SlackConfig, t *template.Template, l *slog.Logger, templater alertmanagertypes.Templater, httpOpts ...commoncfg.HTTPClientOption) (*Notifier, error) {
client, err := notify.NewClientWithTracing(*c.HTTPConfig, Integration, httpOpts...)
if err != nil {
return nil, err
@@ -54,6 +61,7 @@ func New(c *config.SlackConfig, t *template.Template, l *slog.Logger, httpOpts .
logger: l,
client: client,
retrier: &notify.Retrier{},
templater: templater,
postJSONFunc: notify.PostJSON,
}, nil
}
@@ -81,9 +89,10 @@ type attachment struct {
Actions []config.SlackAction `json:"actions,omitempty"`
ImageURL string `json:"image_url,omitempty"`
ThumbURL string `json:"thumb_url,omitempty"`
Footer string `json:"footer"`
Footer string `json:"footer,omitempty"`
Color string `json:"color,omitempty"`
MrkdwnIn []string `json:"mrkdwn_in,omitempty"`
Blocks []any `json:"blocks,omitempty"`
}
// Notify implements the Notifier interface.
@@ -100,79 +109,15 @@ func (n *Notifier) Notify(ctx context.Context, as ...*types.Alert) (bool, error)
data = notify.GetTemplateData(ctx, n.tmpl, as, logger)
tmplText = notify.TmplText(n.tmpl, data, &err)
)
var markdownIn []string
if len(n.conf.MrkdwnIn) == 0 {
markdownIn = []string{"fallback", "pretext", "text"}
} else {
markdownIn = n.conf.MrkdwnIn
attachments, err := n.prepareContent(ctx, as, tmplText)
if err != nil {
n.logger.ErrorContext(ctx, "failed to prepare notification content", errors.Attr(err))
return false, err
}
title, truncated := notify.TruncateInRunes(tmplText(n.conf.Title), maxTitleLenRunes)
if truncated {
logger.WarnContext(ctx, "Truncated title", slog.Int("max_runes", maxTitleLenRunes))
}
att := &attachment{
Title: title,
TitleLink: tmplText(n.conf.TitleLink),
Pretext: tmplText(n.conf.Pretext),
Text: tmplText(n.conf.Text),
Fallback: tmplText(n.conf.Fallback),
CallbackID: tmplText(n.conf.CallbackID),
ImageURL: tmplText(n.conf.ImageURL),
ThumbURL: tmplText(n.conf.ThumbURL),
Footer: tmplText(n.conf.Footer),
Color: tmplText(n.conf.Color),
MrkdwnIn: markdownIn,
}
numFields := len(n.conf.Fields)
if numFields > 0 {
fields := make([]config.SlackField, numFields)
for index, field := range n.conf.Fields {
// Check if short was defined for the field otherwise fallback to the global setting
var short bool
if field.Short != nil {
short = *field.Short
} else {
short = n.conf.ShortFields
}
// Rebuild the field by executing any templates and setting the new value for short
fields[index] = config.SlackField{
Title: tmplText(field.Title),
Value: tmplText(field.Value),
Short: &short,
}
}
att.Fields = fields
}
numActions := len(n.conf.Actions)
if numActions > 0 {
actions := make([]config.SlackAction, numActions)
for index, action := range n.conf.Actions {
slackAction := config.SlackAction{
Type: tmplText(action.Type),
Text: tmplText(action.Text),
URL: tmplText(action.URL),
Style: tmplText(action.Style),
Name: tmplText(action.Name),
Value: tmplText(action.Value),
}
if action.ConfirmField != nil {
slackAction.ConfirmField = &config.SlackConfirmationField{
Title: tmplText(action.ConfirmField.Title),
Text: tmplText(action.ConfirmField.Text),
OkText: tmplText(action.ConfirmField.OkText),
DismissText: tmplText(action.ConfirmField.DismissText),
}
}
actions[index] = slackAction
}
att.Actions = actions
if len(attachments) > 0 {
n.addFieldsAndActions(&attachments[0], tmplText)
}
req := &request{
@@ -182,7 +127,7 @@ func (n *Notifier) Notify(ctx context.Context, as ...*types.Alert) (bool, error)
IconURL: tmplText(n.conf.IconURL),
LinkNames: n.conf.LinkNames,
Text: tmplText(n.conf.MessageText),
Attachments: []attachment{*att},
Attachments: attachments,
}
if err != nil {
return false, err
@@ -238,6 +183,150 @@ func (n *Notifier) Notify(ctx context.Context, as ...*types.Alert) (bool, error)
return retry, nil
}
// prepareContent expands alert templates and returns the Slack attachment(s)
// ready to send. When alerts carry a custom body template, one title-only
// attachment plus one body attachment per alert is returned so that each alert
// can get its own firing/resolved color and per-alert action buttons.
func (n *Notifier) prepareContent(ctx context.Context, alerts []*types.Alert, tmplText func(string) string) ([]attachment, error) {
customTitle, customBody := alertmanagertemplate.ExtractTemplatesFromAnnotations(alerts)
result, err := n.templater.Expand(ctx, alertmanagertypes.ExpandRequest{
TitleTemplate: customTitle,
BodyTemplate: customBody,
DefaultTitleTemplate: n.conf.Title,
DefaultBodyTemplate: n.conf.Text,
}, alerts)
if err != nil {
return nil, err
}
title, truncated := notify.TruncateInRunes(result.Title, maxTitleLenRunes)
if truncated {
n.logger.WarnContext(ctx, "Truncated title", slog.Int("max_runes", maxTitleLenRunes))
}
if result.IsDefaultBody {
var markdownIn []string
if len(n.conf.MrkdwnIn) == 0 {
markdownIn = []string{"fallback", "pretext", "text"}
} else {
markdownIn = n.conf.MrkdwnIn
}
return []attachment{
{
Title: title,
TitleLink: tmplText(n.conf.TitleLink),
Pretext: tmplText(n.conf.Pretext),
Text: result.Body[0],
Fallback: tmplText(n.conf.Fallback),
CallbackID: tmplText(n.conf.CallbackID),
ImageURL: tmplText(n.conf.ImageURL),
ThumbURL: tmplText(n.conf.ThumbURL),
Footer: tmplText(n.conf.Footer),
Color: tmplText(n.conf.Color),
MrkdwnIn: markdownIn,
},
}, nil
}
// Custom template path: one title attachment + one attachment per
// non-empty alert body. result.Body is positionally aligned with alerts,
// so we index alerts[i] directly and skip empty entries.
attachments := make([]attachment, 0, 1+len(result.Body))
attachments = append(attachments, attachment{
Title: title,
TitleLink: tmplText(n.conf.TitleLink),
})
for i, body := range result.Body {
if body == "" || i >= len(alerts) {
continue
}
// Custom bodies are authored in markdown; render each non-empty body to
// Slack's mrkdwn flavour. Default bodies skip this because the Text
// template is already channel-ready.
rendered, renderErr := markdownrenderer.RenderSlackMrkdwn(body)
if renderErr != nil {
return nil, renderErr
}
color := colorRed
if alerts[i].Resolved() {
color = colorGreen
}
attachments = append(attachments, attachment{
Text: rendered,
Color: color,
MrkdwnIn: []string{"text"},
Actions: buildRelatedLinkActions(alerts[i]),
})
}
return attachments, nil
}
// buildRelatedLinkActions returns the "View Related Logs/Traces" action
// buttons for an alert, or nil when no related-link annotations are present.
func buildRelatedLinkActions(alert *types.Alert) []config.SlackAction {
var actions []config.SlackAction
if link := alert.Annotations[ruletypes.AnnotationRelatedLogs]; link != "" {
actions = append(actions, config.SlackAction{Type: "button", Text: "View Related Logs", URL: string(link)})
}
if link := alert.Annotations[ruletypes.AnnotationRelatedTraces]; link != "" {
actions = append(actions, config.SlackAction{Type: "button", Text: "View Related Traces", URL: string(link)})
}
return actions
}
// addFieldsAndActions populates fields and actions on the attachment from the Slack config.
func (n *Notifier) addFieldsAndActions(att *attachment, tmplText func(string) string) {
numFields := len(n.conf.Fields)
if numFields > 0 {
fields := make([]config.SlackField, numFields)
for index, field := range n.conf.Fields {
var short bool
if field.Short != nil {
short = *field.Short
} else {
short = n.conf.ShortFields
}
fields[index] = config.SlackField{
Title: tmplText(field.Title),
Value: tmplText(field.Value),
Short: &short,
}
}
att.Fields = fields
}
numActions := len(n.conf.Actions)
if numActions > 0 {
actions := make([]config.SlackAction, numActions)
for index, action := range n.conf.Actions {
slackAction := config.SlackAction{
Type: tmplText(action.Type),
Text: tmplText(action.Text),
URL: tmplText(action.URL),
Style: tmplText(action.Style),
Name: tmplText(action.Name),
Value: tmplText(action.Value),
}
if action.ConfirmField != nil {
slackAction.ConfirmField = &config.SlackConfirmationField{
Title: tmplText(action.ConfirmField.Title),
Text: tmplText(action.ConfirmField.Text),
OkText: tmplText(action.ConfirmField.OkText),
DismissText: tmplText(action.ConfirmField.DismissText),
}
}
actions[index] = slackAction
}
att.Actions = actions
}
}
// checkResponseError parses out the error message from Slack API response.
func checkResponseError(resp *http.Response) (bool, error) {
body, err := io.ReadAll(resp.Body)

View File

@@ -17,6 +17,9 @@ import (
"testing"
"time"
"github.com/SigNoz/signoz/pkg/alertmanager/alertmanagertemplate"
"github.com/SigNoz/signoz/pkg/types/alertmanagertypes"
"github.com/SigNoz/signoz/pkg/types/ruletypes"
commoncfg "github.com/prometheus/common/config"
"github.com/prometheus/common/model"
"github.com/prometheus/common/promslog"
@@ -29,13 +32,19 @@ import (
"github.com/prometheus/alertmanager/types"
)
func newTestTemplater(tmpl *template.Template) alertmanagertypes.Templater {
return alertmanagertemplate.New(tmpl, slog.New(slog.DiscardHandler))
}
func TestSlackRetry(t *testing.T) {
tmpl := test.CreateTmpl(t)
notifier, err := New(
&config.SlackConfig{
HTTPConfig: &commoncfg.HTTPClientConfig{},
},
test.CreateTmpl(t),
tmpl,
promslog.NewNopLogger(),
newTestTemplater(tmpl),
)
require.NoError(t, err)
@@ -49,13 +58,15 @@ func TestSlackRedactedURL(t *testing.T) {
ctx, u, fn := test.GetContextWithCancelingURL()
defer fn()
tmpl := test.CreateTmpl(t)
notifier, err := New(
&config.SlackConfig{
APIURL: &config.SecretURL{URL: u},
HTTPConfig: &commoncfg.HTTPClientConfig{},
},
test.CreateTmpl(t),
tmpl,
promslog.NewNopLogger(),
newTestTemplater(tmpl),
)
require.NoError(t, err)
@@ -71,13 +82,15 @@ func TestGettingSlackURLFromFile(t *testing.T) {
_, err = f.WriteString(u.String())
require.NoError(t, err, "writing to temp file failed")
tmpl := test.CreateTmpl(t)
notifier, err := New(
&config.SlackConfig{
APIURLFile: f.Name(),
HTTPConfig: &commoncfg.HTTPClientConfig{},
},
test.CreateTmpl(t),
tmpl,
promslog.NewNopLogger(),
newTestTemplater(tmpl),
)
require.NoError(t, err)
@@ -93,13 +106,15 @@ func TestTrimmingSlackURLFromFile(t *testing.T) {
_, err = f.WriteString(u.String() + "\n\n")
require.NoError(t, err, "writing to temp file failed")
tmpl := test.CreateTmpl(t)
notifier, err := New(
&config.SlackConfig{
APIURLFile: f.Name(),
HTTPConfig: &commoncfg.HTTPClientConfig{},
},
test.CreateTmpl(t),
tmpl,
promslog.NewNopLogger(),
newTestTemplater(tmpl),
)
require.NoError(t, err)
@@ -184,6 +199,7 @@ func TestNotifier_Notify_WithReason(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
apiurl, _ := url.Parse("https://slack.com/post.Message")
tmpl := test.CreateTmpl(t)
notifier, err := New(
&config.SlackConfig{
NotifierConfig: config.NotifierConfig{},
@@ -191,8 +207,9 @@ func TestNotifier_Notify_WithReason(t *testing.T) {
APIURL: &config.SecretURL{URL: apiurl},
Channel: "channelname",
},
test.CreateTmpl(t),
tmpl,
promslog.NewNopLogger(),
newTestTemplater(tmpl),
)
require.NoError(t, err)
@@ -242,6 +259,7 @@ func TestSlackTimeout(t *testing.T) {
for name, tt := range tests {
t.Run(name, func(t *testing.T) {
u, _ := url.Parse("https://slack.com/post.Message")
tmpl := test.CreateTmpl(t)
notifier, err := New(
&config.SlackConfig{
NotifierConfig: config.NotifierConfig{},
@@ -250,8 +268,9 @@ func TestSlackTimeout(t *testing.T) {
Channel: "channelname",
Timeout: tt.timeout,
},
test.CreateTmpl(t),
tmpl,
promslog.NewNopLogger(),
newTestTemplater(tmpl),
)
require.NoError(t, err)
notifier.postJSONFunc = func(ctx context.Context, client *http.Client, url string, body io.Reader) (*http.Response, error) {
@@ -282,6 +301,225 @@ func TestSlackTimeout(t *testing.T) {
}
}
// setupTestContext creates a context with group key, receiver name, and group labels
// required by the notification processor.
func setupTestContext() context.Context {
ctx := context.Background()
ctx = notify.WithGroupKey(ctx, "test-group")
ctx = notify.WithReceiverName(ctx, "slack")
ctx = notify.WithGroupLabels(ctx, model.LabelSet{
"alertname": "TestAlert",
"severity": "critical",
})
return ctx
}
func TestPrepareContent(t *testing.T) {
t.Run("default template uses go text template config for title and body", func(t *testing.T) {
// When alerts have no custom annotation templates (title_template / body_template),
tmpl := test.CreateTmpl(t)
templater := newTestTemplater(tmpl)
notifier := &Notifier{
conf: &config.SlackConfig{
Title: `{{ .CommonLabels.alertname }} ({{ .Status | toUpper }})`,
Text: `{{ range .Alerts }}Alert: {{ .Labels.alertname }} - severity {{ .Labels.severity }}{{ end }}`,
Color: `{{ if eq .Status "firing" }}danger{{ else }}good{{ end }}`,
TitleLink: "https://alertmanager.signoz.com",
},
tmpl: tmpl,
logger: slog.New(slog.DiscardHandler),
templater: templater,
}
ctx := setupTestContext()
alerts := []*types.Alert{
{Alert: model.Alert{
Labels: model.LabelSet{ruletypes.LabelAlertName: "HighCPU", ruletypes.LabelSeverityName: "critical"},
StartsAt: time.Now(),
EndsAt: time.Now().Add(time.Hour),
}},
}
// Build tmplText the same way Notify does
var err error
data := notify.GetTemplateData(ctx, tmpl, alerts, slog.New(slog.DiscardHandler))
tmplText := notify.TmplText(tmpl, data, &err)
atts, attErr := notifier.prepareContent(ctx, alerts, tmplText)
require.NoError(t, attErr)
require.NoError(t, err)
require.Len(t, atts, 1)
require.Equal(t, "HighCPU (FIRING)", atts[0].Title)
require.Equal(t, "Alert: HighCPU - severity critical", atts[0].Text)
// Color is templated — firing alert should be "danger"
require.Equal(t, "danger", atts[0].Color)
// No BlockKit blocks for default template
require.Nil(t, atts[0].Blocks)
// Default markdownIn when config has none
require.Equal(t, []string{"fallback", "pretext", "text"}, atts[0].MrkdwnIn)
})
t.Run("custom template produces 1+N attachments with per-alert color", func(t *testing.T) {
// When alerts carry custom $variable annotation templates (title_template / body_template)
tmpl := test.CreateTmpl(t)
templater := newTestTemplater(tmpl)
notifier := &Notifier{
conf: &config.SlackConfig{
Title: "default title fallback",
Text: "default text fallback",
TitleLink: "https://alertmanager.signoz.com",
},
tmpl: tmpl,
logger: slog.New(slog.DiscardHandler),
templater: templater,
}
tmplText := func(s string) string { return s }
bodyTemplate := `## $rule.name
**Service:** *$labels.service*
**Instance:** *$labels.instance*
**Region:** *$labels.region*
**Method:** *$labels.http_method*
---
| Metric | Value |
|--------|-------|
| **Current** | *$value* |
| **Threshold** | *$threshold.value* |
**Status:** $alert.status | **Severity:** $labels.severity`
titleTemplate := "[$alert.status] $rule.name — $labels.service"
ctx := setupTestContext()
firingAlert := &types.Alert{
Alert: model.Alert{
Labels: model.LabelSet{ruletypes.LabelAlertName: "HighCPU", ruletypes.LabelSeverityName: "critical", "service": "api-server", "instance": "i-0abc123", "region": "us-east-1", "http_method": "GET"},
StartsAt: time.Now(),
EndsAt: time.Now().Add(time.Hour),
Annotations: model.LabelSet{
ruletypes.AnnotationTitleTemplate: model.LabelValue(titleTemplate),
ruletypes.AnnotationBodyTemplate: model.LabelValue(bodyTemplate),
"value": "100",
"threshold.value": "200",
},
},
}
resolvedAlert := &types.Alert{
Alert: model.Alert{
Labels: model.LabelSet{ruletypes.LabelAlertName: "HighCPU", ruletypes.LabelSeverityName: "critical", "service": "api-server", "instance": "i-0abc123", "region": "us-east-1", "http_method": "GET"},
StartsAt: time.Now().Add(-2 * time.Hour),
EndsAt: time.Now().Add(-time.Hour),
Annotations: model.LabelSet{
ruletypes.AnnotationTitleTemplate: model.LabelValue(titleTemplate),
ruletypes.AnnotationBodyTemplate: model.LabelValue(bodyTemplate),
"value": "50",
"threshold.value": "200",
},
},
}
atts, err := notifier.prepareContent(ctx, []*types.Alert{firingAlert, resolvedAlert}, tmplText)
require.NoError(t, err)
// 1 title attachment + 2 body attachments (one per alert)
require.Len(t, atts, 3)
// First attachment: title-only, no color, no blocks
require.Equal(t, "[firing] HighCPU — api-server", atts[0].Title)
require.Empty(t, atts[0].Color)
require.Nil(t, atts[0].Blocks)
require.Equal(t, "https://alertmanager.signoz.com", atts[0].TitleLink)
expectedFiringBody := "*HighCPU*\n\n" +
"*Service:* _api-server_\n*Instance:* _i-0abc123_\n*Region:* _us-east-1_\n*Method:* _GET_\n\n" +
"---\n\n" +
"```\nMetric | Value\n----------|------\nCurrent | 100 \nThreshold | 200 \n```\n\n" +
"*Status:* firing | *Severity:* critical\n\n"
expectedResolvedBody := "*HighCPU*\n\n" +
"*Service:* _api-server_\n*Instance:* _i-0abc123_\n*Region:* _us-east-1_\n*Method:* _GET_\n\n" +
"---\n\n" +
"```\nMetric | Value\n----------|------\nCurrent | 50 \nThreshold | 200 \n```\n\n" +
"*Status:* resolved | *Severity:* critical\n\n"
// Second attachment: firing alert body rendered as slack mrkdwn text, red color
require.Nil(t, atts[1].Blocks)
require.Equal(t, "#FF0000", atts[1].Color)
require.Equal(t, []string{"text"}, atts[1].MrkdwnIn)
require.Equal(t, expectedFiringBody, atts[1].Text)
// Third attachment: resolved alert body rendered as slack mrkdwn text, green color
require.Nil(t, atts[2].Blocks)
require.Equal(t, "#00FF00", atts[2].Color)
require.Equal(t, []string{"text"}, atts[2].MrkdwnIn)
require.Equal(t, expectedResolvedBody, atts[2].Text)
})
t.Run("default template with fields and actions", func(t *testing.T) {
// Verifies that addFieldsAndActions (called from Notify after prepareContent)
// correctly populates fields and actions on the attachment from config.
tmpl := test.CreateTmpl(t)
templater := newTestTemplater(tmpl)
short := true
notifier := &Notifier{
conf: &config.SlackConfig{
Title: `{{ .CommonLabels.alertname }}`,
Text: "alert text",
Color: "warning",
Fields: []*config.SlackField{
{Title: "Severity", Value: "critical", Short: &short},
{Title: "Service", Value: "api-server", Short: &short},
},
Actions: []*config.SlackAction{
{Type: "button", Text: "View Alert", URL: "https://alertmanager.signoz.com"},
},
TitleLink: "https://alertmanager.signoz.com",
},
tmpl: tmpl,
logger: slog.New(slog.DiscardHandler),
templater: templater,
}
tmplText := func(s string) string { return s }
ctx := setupTestContext()
alerts := []*types.Alert{
{Alert: model.Alert{
Labels: model.LabelSet{ruletypes.LabelAlertName: "TestAlert"},
StartsAt: time.Now(),
EndsAt: time.Now().Add(time.Hour),
}},
}
atts, err := notifier.prepareContent(ctx, alerts, tmplText)
require.NoError(t, err)
require.Len(t, atts, 1)
// prepareContent does not populate fields/actions — that's done by
// addFieldsAndActions which is called from Notify.
require.Nil(t, atts[0].Fields)
require.Nil(t, atts[0].Actions)
// Simulate what Notify does after prepareContent
notifier.addFieldsAndActions(&atts[0], tmplText)
// Verify fields
require.Len(t, atts[0].Fields, 2)
require.Equal(t, "Severity", atts[0].Fields[0].Title)
require.Equal(t, "critical", atts[0].Fields[0].Value)
require.True(t, *atts[0].Fields[0].Short)
require.Equal(t, "Service", atts[0].Fields[1].Title)
require.Equal(t, "api-server", atts[0].Fields[1].Value)
// Verify actions
require.Len(t, atts[0].Actions, 1)
require.Equal(t, "button", atts[0].Actions[0].Type)
require.Equal(t, "View Alert", atts[0].Actions[0].Text)
require.Equal(t, "https://alertmanager.signoz.com", atts[0].Actions[0].URL)
})
}
func TestSlackMessageField(t *testing.T) {
// 1. Setup a fake Slack server
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
@@ -329,7 +567,7 @@ func TestSlackMessageField(t *testing.T) {
tmpl.ExternalURL = u
logger := slog.New(slog.DiscardHandler)
notifier, err := New(conf, tmpl, logger)
notifier, err := New(conf, tmpl, logger, newTestTemplater(tmpl))
if err != nil {
t.Fatal(err)
}

View File

@@ -14,6 +14,7 @@ import (
"strings"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types/alertmanagertypes"
commoncfg "github.com/prometheus/common/config"
"github.com/prometheus/alertmanager/config"
@@ -28,15 +29,16 @@ const (
// Notifier implements a Notifier for generic webhooks.
type Notifier struct {
conf *config.WebhookConfig
tmpl *template.Template
logger *slog.Logger
client *http.Client
retrier *notify.Retrier
conf *config.WebhookConfig
tmpl *template.Template
logger *slog.Logger
client *http.Client
retrier *notify.Retrier
templater alertmanagertypes.Templater
}
// New returns a new Webhook.
func New(conf *config.WebhookConfig, t *template.Template, l *slog.Logger, httpOpts ...commoncfg.HTTPClientOption) (*Notifier, error) {
func New(conf *config.WebhookConfig, t *template.Template, l *slog.Logger, templater alertmanagertypes.Templater, httpOpts ...commoncfg.HTTPClientOption) (*Notifier, error) {
client, err := notify.NewClientWithTracing(*conf.HTTPConfig, Integration, httpOpts...)
if err != nil {
return nil, err
@@ -48,7 +50,8 @@ func New(conf *config.WebhookConfig, t *template.Template, l *slog.Logger, httpO
client: client,
// Webhooks are assumed to respond with 2xx response codes on a successful
// request and 5xx response codes are assumed to be recoverable.
retrier: &notify.Retrier{},
retrier: &notify.Retrier{},
templater: templater,
}, nil
}

View File

@@ -9,6 +9,7 @@ import (
"context"
"fmt"
"io"
"log/slog"
"net/http"
"net/http/httptest"
"os"
@@ -20,6 +21,7 @@ import (
"github.com/prometheus/common/promslog"
"github.com/stretchr/testify/require"
"github.com/SigNoz/signoz/pkg/alertmanager/alertmanagertemplate"
"github.com/prometheus/alertmanager/config"
"github.com/prometheus/alertmanager/notify"
"github.com/prometheus/alertmanager/notify/test"
@@ -27,13 +29,15 @@ import (
)
func TestWebhookRetry(t *testing.T) {
tmpl := test.CreateTmpl(t)
notifier, err := New(
&config.WebhookConfig{
URL: config.SecretTemplateURL("http://example.com"),
HTTPConfig: &commoncfg.HTTPClientConfig{},
},
test.CreateTmpl(t),
tmpl,
promslog.NewNopLogger(),
alertmanagertemplate.New(tmpl, slog.Default()),
)
if err != nil {
require.NoError(t, err)
@@ -96,13 +100,16 @@ func TestWebhookRedactedURL(t *testing.T) {
defer fn()
secret := "secret"
tmpl := test.CreateTmpl(t)
templater := alertmanagertemplate.New(tmpl, slog.Default())
notifier, err := New(
&config.WebhookConfig{
URL: config.SecretTemplateURL(u.String()),
HTTPConfig: &commoncfg.HTTPClientConfig{},
},
test.CreateTmpl(t),
tmpl,
promslog.NewNopLogger(),
templater,
)
require.NoError(t, err)
@@ -118,13 +125,15 @@ func TestWebhookReadingURLFromFile(t *testing.T) {
_, err = f.WriteString(u.String() + "\n")
require.NoError(t, err, "writing to temp file failed")
tmpl := test.CreateTmpl(t)
notifier, err := New(
&config.WebhookConfig{
URLFile: f.Name(),
HTTPConfig: &commoncfg.HTTPClientConfig{},
},
test.CreateTmpl(t),
tmpl,
promslog.NewNopLogger(),
alertmanagertemplate.New(tmpl, slog.Default()),
)
require.NoError(t, err)
@@ -178,13 +187,15 @@ func TestWebhookURLTemplating(t *testing.T) {
t.Run(tc.name, func(t *testing.T) {
calledURL = "" // Reset for each test
tmpl := test.CreateTmpl(t)
notifier, err := New(
&config.WebhookConfig{
URL: config.SecretTemplateURL(tc.url),
HTTPConfig: &commoncfg.HTTPClientConfig{},
},
test.CreateTmpl(t),
tmpl,
promslog.NewNopLogger(),
alertmanagertemplate.New(tmpl, slog.Default()),
)
require.NoError(t, err)

View File

@@ -28,6 +28,13 @@ type Config struct {
// Configuration for the notification log.
NFLog NFLogConfig `mapstructure:"nflog"`
// Templates is the list of globs from which SigNoz's alertmanager notification
// templates are loaded (e.g. the email.signoz.html layout). This mirrors the
// upstream alertmanager `templates` config option (https://github.com/prometheus/alertmanager/blob/3b06b97af4d146e141af92885a185891eb79a5b0/config/config.go#L412).
// The upstream default templates (default.tmpl, email.tmpl) are always loaded
// from the embedded alertmanager assets, so only SigNoz's own templates are listed here.
Templates []string `mapstructure:"templates"`
}
type AlertsConfig struct {
@@ -100,5 +107,6 @@ func NewConfig() Config {
MaintenanceInterval: 15 * time.Minute,
Retention: 120 * time.Hour,
},
Templates: []string{"/root/templates/alertmanager/*.gotmpl"},
}
}

View File

@@ -42,7 +42,14 @@ func (m *MaintenanceMuter) Mutes(ctx context.Context, lset model.LabelSet) bool
}
now := time.Now()
for _, mw := range m.getMaintenances(ctx) {
if mw.ShouldSkip(ruleID, now) {
skip, err := mw.ShouldSkip(ruleID, now, lset)
if err != nil {
m.logger.ErrorContext(ctx, "failed to test maintenance window skip condition",
slog.String("maintenance_id", mw.ID.StringValue()),
slog.String("scope", mw.Scope),
slog.Any("error", err),
)
} else if skip {
return true
}
}
@@ -61,7 +68,14 @@ func (m *MaintenanceMuter) MutedBy(ctx context.Context, lset model.LabelSet) []s
var ids []string
now := time.Now()
for _, mw := range m.getMaintenances(ctx) {
if mw.ShouldSkip(ruleID, now) {
skip, err := mw.ShouldSkip(ruleID, now, lset)
if err != nil {
m.logger.ErrorContext(ctx, "failed to test maintenance window skip condition",
slog.String("maintenance_id", mw.ID.StringValue()),
slog.String("scope", mw.Scope),
slog.Any("error", err),
)
} else if skip {
ids = append(ids, mw.ID.String())
}
}

View File

@@ -10,6 +10,7 @@ import (
"github.com/prometheus/alertmanager/types"
"golang.org/x/sync/errgroup"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/prometheus/alertmanager/dispatch"
"github.com/prometheus/alertmanager/featurecontrol"
"github.com/prometheus/alertmanager/inhibit"
@@ -23,8 +24,8 @@ import (
"github.com/prometheus/common/model"
"github.com/SigNoz/signoz/pkg/alertmanager/alertmanagernotify"
"github.com/SigNoz/signoz/pkg/alertmanager/alertmanagertemplate"
"github.com/SigNoz/signoz/pkg/alertmanager/nfmanager"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types/alertmanagertypes"
)
@@ -65,6 +66,7 @@ type Server struct {
muter *MaintenanceMuter
marker *types.MemMarker
tmpl *template.Template
templater alertmanagertypes.Templater
wg sync.WaitGroup
stopc chan struct{}
notificationManager nfmanager.NotificationManager
@@ -242,13 +244,21 @@ func (server *Server) SetConfig(ctx context.Context, alertmanagerConfig *alertma
config := alertmanagerConfig.AlertmanagerConfig()
var err error
server.tmpl, err = alertmanagertypes.FromGlobs(config.Templates)
// Load SigNoz's alertmanager notification templates from the configured
// globs. The upstream default templates (default.tmpl, email.tmpl) are
// always loaded from the embedded alertmanager assets inside FromGlobs, so
// only SigNoz's own templates (e.g. the email.signoz.html layout) are listed
// here. The upstream config.Templates field is not used: SigNoz never
// populates it (there is no per-org template configuration).
server.tmpl, err = alertmanagertypes.FromGlobs(server.srvConfig.Templates)
if err != nil {
return err
}
server.tmpl.ExternalURL = server.srvConfig.ExternalURL
server.templater = alertmanagertemplate.New(server.tmpl, server.logger)
// Build the routing tree and record which receivers are used.
routes := dispatch.NewRoute(config.Route, nil)
activeReceivers := make(map[string]struct{})
@@ -265,7 +275,7 @@ func (server *Server) SetConfig(ctx context.Context, alertmanagerConfig *alertma
server.logger.InfoContext(ctx, "skipping creation of receiver not referenced by any route", slog.String("receiver", rcv.Name))
continue
}
integrations, err := alertmanagernotify.NewReceiverIntegrations(rcv, server.tmpl, server.logger)
integrations, err := alertmanagernotify.NewReceiverIntegrations(rcv, server.tmpl, server.logger, server.templater)
if err != nil {
return err
}
@@ -342,7 +352,7 @@ func (server *Server) SetConfig(ctx context.Context, alertmanagerConfig *alertma
func (server *Server) TestReceiver(ctx context.Context, receiver alertmanagertypes.Receiver) error {
testAlert := alertmanagertypes.NewTestAlert(receiver, time.Now(), time.Now())
return alertmanagertypes.TestReceiver(ctx, receiver, alertmanagernotify.NewReceiverIntegrations, server.alertmanagerConfig, server.tmpl, server.logger, testAlert.Labels, testAlert)
return alertmanagertypes.TestReceiver(ctx, receiver, alertmanagernotify.NewReceiverIntegrations, server.alertmanagerConfig, server.tmpl, server.logger, server.templater, testAlert.Labels, testAlert)
}
func (server *Server) TestAlert(ctx context.Context, receiversMap map[*alertmanagertypes.PostableAlert][]string, config *alertmanagertypes.NotificationConfig) error {
@@ -425,6 +435,7 @@ func (server *Server) TestAlert(ctx context.Context, receiversMap map[*alertmana
server.alertmanagerConfig,
server.tmpl,
server.logger,
server.templater,
group.groupLabels,
group.alerts...,
)

View File

@@ -41,7 +41,7 @@ func TestEndToEndAlertManagerFlow(t *testing.T) {
Identifiable: types.Identifiable{
ID: valuer.GenerateUUID(),
},
Expression: `ruleId == "high-cpu-usage" && severity == "critical"`,
Expression: `ruleId = "high-cpu-usage" AND severity = "critical"`,
ExpressionKind: alertmanagertypes.RuleBasedExpression,
Name: "high-cpu-usage",
Description: "High CPU critical alerts to webhook",
@@ -53,7 +53,7 @@ func TestEndToEndAlertManagerFlow(t *testing.T) {
Identifiable: types.Identifiable{
ID: valuer.GenerateUUID(),
},
Expression: `ruleId == "high-cpu-usage" && severity == "warning"`,
Expression: `ruleId = "high-cpu-usage" AND severity = "warning"`,
ExpressionKind: alertmanagertypes.RuleBasedExpression,
Name: "high-cpu-usage",
Description: "High CPU warning alerts to webhook",
@@ -87,18 +87,25 @@ func TestEndToEndAlertManagerFlow(t *testing.T) {
err = notificationManager.SetNotificationConfig(orgID, "high-cpu-usage", &notifConfig)
require.NoError(t, err)
mwID := valuer.GenerateUUID()
activeSchedule := &alertmanagertypes.Schedule{
Timezone: "UTC",
StartTime: time.Now().Add(-time.Hour),
EndTime: time.Now().Add(time.Hour),
}
// mwRuleIDAndScope: only critical high-cpu-usage alerts.
mwRuleIDAndScope := valuer.GenerateUUID()
// mwRuleIDOnly: all high-cpu-usage alerts regardless of severity.
mwRuleIDOnly := valuer.GenerateUUID()
// mwScopeOnly: all critical alerts regardless of rule ID.
mwScopeOnly := valuer.GenerateUUID()
maintenanceStore := alertmanagertypestest.NewMockMaintenanceStore(t)
maintenanceStore.On("ListPlannedMaintenance", mock.Anything, orgID).Return(
[]*alertmanagertypes.PlannedMaintenance{{
ID: mwID,
Schedule: &alertmanagertypes.Schedule{
Timezone: "UTC",
StartTime: time.Now().Add(-time.Hour),
EndTime: time.Now().Add(time.Hour),
},
RuleIDs: []string{"high-cpu-usage"},
}}, nil,
[]*alertmanagertypes.PlannedMaintenance{
{ID: mwRuleIDAndScope, Schedule: activeSchedule, RuleIDs: []string{"high-cpu-usage"}, Scope: `severity = "critical"`},
{ID: mwRuleIDOnly, Schedule: activeSchedule, RuleIDs: []string{"high-cpu-usage"}},
{ID: mwScopeOnly, Schedule: activeSchedule, Scope: `severity = "critical"`},
}, nil,
)
srvCfg := NewConfig()
@@ -249,18 +256,42 @@ func TestEndToEndAlertManagerFlow(t *testing.T) {
require.Equal(t, "{__receiver__=\"webhook\"}:{cluster=\"prod-cluster\", instance=\"server-03\", ruleId=\"high-cpu-usage\"}", alertGroups[2].GroupKey)
})
t.Run("verify_muting", func(t *testing.T) {
req, err := http.NewRequest(http.MethodGet, "/alerts", nil)
require.NoError(t, err)
params, err := alertmanagertypes.NewGettableAlertsParams(req)
require.NoError(t, err)
alerts, err := server.GetAlerts(ctx, params)
require.NoError(t, err)
req, err := http.NewRequest(http.MethodGet, "/alerts", nil)
require.NoError(t, err)
params, err := alertmanagertypes.NewGettableAlertsParams(req)
require.NoError(t, err)
alerts, err := server.GetAlerts(ctx, params)
require.NoError(t, err)
t.Run("verify_muting_ruleid_and_scope", func(t *testing.T) {
// Window with ruleID + scope mutes only alerts matching both.
for _, alert := range alerts {
if alert.Labels["ruleId"] == "high-cpu-usage" && alert.Labels["severity"] == "critical" {
require.Contains(t, alert.Status.MutedBy, mwRuleIDAndScope.String())
} else {
require.NotContains(t, alert.Status.MutedBy, mwRuleIDAndScope.String())
}
}
})
t.Run("verify_muting_ruleid_only", func(t *testing.T) {
// Window with ruleID but no scope mutes all severities for that rule.
for _, alert := range alerts {
if alert.Labels["ruleId"] == "high-cpu-usage" {
require.Equal(t, []string{mwID.String()}, alert.Status.MutedBy)
require.Contains(t, alert.Status.MutedBy, mwRuleIDOnly.String())
} else {
require.Empty(t, alert.Status.MutedBy)
require.NotContains(t, alert.Status.MutedBy, mwRuleIDOnly.String())
}
}
})
t.Run("verify_muting_scope_only", func(t *testing.T) {
// Window with scope but no ruleIDs mutes all critical alerts regardless of rule.
for _, alert := range alerts {
if alert.Labels["severity"] == "critical" {
require.Contains(t, alert.Status.MutedBy, mwScopeOnly.String())
} else {
require.NotContains(t, alert.Status.MutedBy, mwScopeOnly.String())
}
}
})

View File

@@ -89,6 +89,7 @@ func (r *maintenance) CreatePlannedMaintenance(ctx context.Context, maintenance
Description: maintenance.Description,
Schedule: maintenance.Schedule,
OrgID: claims.OrgID,
Scope: maintenance.Scope,
}
maintenanceRules := make([]*alertmanagertypes.StorablePlannedMaintenanceRule, 0)
@@ -123,7 +124,6 @@ func (r *maintenance) CreatePlannedMaintenance(ctx context.Context, maintenance
NewInsert().
Model(&maintenanceRules).
Exec(ctx)
if err != nil {
return err
}
@@ -141,6 +141,7 @@ func (r *maintenance) CreatePlannedMaintenance(ctx context.Context, maintenance
Description: storablePlannedMaintenance.Description,
Schedule: storablePlannedMaintenance.Schedule,
RuleIDs: maintenance.AlertIds,
Scope: maintenance.Scope,
CreatedAt: storablePlannedMaintenance.CreatedAt,
CreatedBy: storablePlannedMaintenance.CreatedBy,
UpdatedAt: storablePlannedMaintenance.UpdatedAt,
@@ -189,6 +190,7 @@ func (r *maintenance) UpdatePlannedMaintenance(ctx context.Context, maintenance
Description: maintenance.Description,
Schedule: maintenance.Schedule,
OrgID: claims.OrgID,
Scope: maintenance.Scope,
}
storablePlannedMaintenanceRules := make([]*alertmanagertypes.StorablePlannedMaintenanceRule, 0)
@@ -224,7 +226,6 @@ func (r *maintenance) UpdatePlannedMaintenance(ctx context.Context, maintenance
Model(new(alertmanagertypes.StorablePlannedMaintenanceRule)).
Where("planned_maintenance_id = ?", storablePlannedMaintenance.ID.StringValue()).
Exec(ctx)
if err != nil {
return err
}
@@ -241,7 +242,6 @@ func (r *maintenance) UpdatePlannedMaintenance(ctx context.Context, maintenance
}
return nil
})
if err != nil {
return err

View File

@@ -15,13 +15,6 @@ import (
"github.com/prometheus/common/model"
)
// Templater expands user-authored title and body templates against a group
// of alerts and returns channel-ready strings along with the aggregate data
// a caller might reuse (e.g. to render an email layout around the body).
type Templater interface {
Expand(ctx context.Context, req alertmanagertypes.ExpandRequest, alerts []*types.Alert) (*alertmanagertypes.ExpandResult, error)
}
type templater struct {
tmpl *template.Template
logger *slog.Logger
@@ -29,7 +22,7 @@ type templater struct {
// New returns a Templater bound to the given Prometheus alertmanager
// template and logger.
func New(tmpl *template.Template, logger *slog.Logger) Templater {
func New(tmpl *template.Template, logger *slog.Logger) alertmanagertypes.Templater {
return &templater{tmpl: tmpl, logger: logger}
}
@@ -137,6 +130,9 @@ func (at *templater) expandTitle(
}
// expandBody expands the body template for each individual alert. Returns nil if the template is empty.
// Non-nil results are positionally aligned with ntd.Alerts: sb[i] corresponds to alerts[i], and
// entries for alerts whose template expands to empty are kept as "" so callers can index per-alert
// metadata (related links, firing/resolved color) by the same index.
func (at *templater) expandBody(
bodyTemplate string,
ntd *alertmanagertypes.NotificationTemplateData,
@@ -144,7 +140,7 @@ func (at *templater) expandBody(
if bodyTemplate == "" {
return nil, nil, nil
}
var sb []string
sb := make([]string, len(ntd.Alerts))
missingVars := make(map[string]bool)
for i := range ntd.Alerts {
processRes, err := preProcessTemplateAndData(bodyTemplate, &ntd.Alerts[i])
@@ -155,13 +151,10 @@ func (at *templater) expandBody(
if err != nil {
return nil, nil, errors.NewInvalidInputf(errors.CodeInvalidInput, "failed to execute custom body template: %s", err.Error())
}
// add unknown variables and templated text to the result
for k := range processRes.UnknownVars {
missingVars[k] = true
}
if strings.TrimSpace(part) != "" {
sb = append(sb, strings.TrimSpace(part))
}
sb[i] = strings.TrimSpace(part)
}
return sb, missingVars, nil
}
@@ -189,17 +182,20 @@ func (at *templater) buildNotificationTemplateData(
externalURL = at.tmpl.ExternalURL.String()
}
commonAnnotations := extractCommonKV(alerts, func(a *types.Alert) model.LabelSet { return a.Annotations })
// Raw (including private `_*`) kv first so buildRuleInfo can read the
// private rule annotations. The filtered copies are what ends up
// on the template-visible surfaces.
rawCommonAnnotations := extractCommonKV(alerts, func(a *types.Alert) model.LabelSet { return a.Annotations })
commonLabels := extractCommonKV(alerts, func(a *types.Alert) model.LabelSet { return a.Labels })
// aggregate labels and annotations from all alerts
labels := aggregateKV(alerts, func(a *types.Alert) model.LabelSet { return a.Labels })
annotations := aggregateKV(alerts, func(a *types.Alert) model.LabelSet { return a.Annotations })
// Strip private annotations from surfaces visible to templates or
// notifications; the structured fields on AlertInfo/RuleInfo already hold
// anything a template needs from them.
commonAnnotations = alertmanagertypes.FilterPublicAnnotations(commonAnnotations)
// Strip private annotations from template-visible surfaces; the structured
// fields on AlertInfo/RuleInfo already hold anything a template needs from
// them.
commonAnnotations := alertmanagertypes.FilterPublicAnnotations(rawCommonAnnotations)
annotations = alertmanagertypes.FilterPublicAnnotations(annotations)
// build the alert data slice
@@ -233,7 +229,7 @@ func (at *templater) buildNotificationTemplateData(
TotalFiring: firing,
TotalResolved: resolved,
},
Rule: buildRuleInfo(commonLabels, commonAnnotations),
Rule: buildRuleInfo(commonLabels, rawCommonAnnotations),
GroupLabels: gl,
CommonLabels: commonLabels,
CommonAnnotations: commonAnnotations,

View File

@@ -19,7 +19,7 @@ import (
// testSetup returns an AlertTemplater and a context pre-populated with group key,
// receiver name, and group labels for use in tests.
func testSetup(t *testing.T) (Templater, context.Context) {
func testSetup(t *testing.T) (alertmanagertypes.Templater, context.Context) {
t.Helper()
tmpl := test.CreateTmpl(t)
ctx := context.Background()

View File

@@ -3,7 +3,6 @@ package rulebasednotification
import (
"context"
"log/slog"
"strings"
"sync"
"github.com/SigNoz/signoz/pkg/alertmanager/nfmanager"
@@ -235,65 +234,13 @@ func (r *provider) Match(ctx context.Context, orgID string, ruleID string, set m
return matchedChannels, nil
}
// convertLabelSetToEnv converts a flat label set with dotted keys into a nested map structure for expr env.
// when both a leaf and a deeper nested path exist (e.g. "foo" and "foo.bar"),
// the nested structure takes precedence. That means we will replace an existing leaf at any
// intermediate path with a map so we can materialize the deeper structure.
// TODO(srikanthccv): we need a better solution to handle this, remove the following
// when we update the expr to support dotted keys.
// convertLabelSetToEnv delegates to alertmanagertypes.ConvertLabelSetToEnv and
// logs when a key is a prefix of another (e.g. "foo" alongside "foo.bar").
func (r *provider) convertLabelSetToEnv(ctx context.Context, labelSet model.LabelSet) map[string]interface{} {
env := make(map[string]interface{})
logForReview := false
for lk, lv := range labelSet {
key := strings.TrimSpace(string(lk))
value := string(lv)
if strings.Contains(key, ".") {
parts := strings.Split(key, ".")
current := env
for i, raw := range parts {
part := strings.TrimSpace(raw)
last := i == len(parts)-1
if last {
if _, isMap := current[part].(map[string]interface{}); isMap {
logForReview = true
// deeper structure already exists; do not overwrite.
break
}
current[part] = value
break
}
// ensure a map so we can keep descending.
if nextMap, ok := current[part].(map[string]interface{}); ok {
current = nextMap
continue
}
// if absent or a leaf, replace it with a map.
newMap := make(map[string]interface{})
current[part] = newMap
current = newMap
}
continue
}
// if a map already sits here (due to nested keys), keep the map (nested wins).
if _, isMap := env[key].(map[string]interface{}); isMap {
logForReview = true
continue
}
env[key] = value
}
if logForReview {
env, conflict := alertmanagertypes.ConvertLabelSetToEnv(labelSet)
if conflict {
r.settings.Logger().InfoContext(ctx, "found label set with conflicting prefix dotted keys", slog.Any("labels", labelSet))
}
return env
}

View File

@@ -925,72 +925,3 @@ func TestProvider_CreateRoutes(t *testing.T) {
})
}
}
func TestConvertLabelSetToEnv(t *testing.T) {
tests := []struct {
name string
labelSet model.LabelSet
expected map[string]interface{}
}{
{
name: "simple keys",
labelSet: model.LabelSet{
"key1": "value1",
"key2": "value2",
},
expected: map[string]interface{}{
"key1": "value1",
"key2": "value2",
},
},
{
name: "nested keys",
labelSet: model.LabelSet{
"foo.bar": "value1",
"foo.baz": "value2",
},
expected: map[string]interface{}{
"foo": map[string]interface{}{
"bar": "value1",
"baz": "value2",
},
},
},
{
name: "conflict - nested structure wins",
labelSet: model.LabelSet{
"foo.bar.baz": "deep",
"foo.bar": "shallow",
},
expected: map[string]interface{}{
"foo": map[string]interface{}{
"bar": map[string]interface{}{
"baz": "deep",
},
},
},
},
{
name: "conflict - leaf value vs nested",
labelSet: model.LabelSet{
"foo.bar": "value",
"foo": "should_be_ignored",
},
expected: map[string]interface{}{
"foo": map[string]interface{}{
"bar": "value",
},
},
},
}
provider := &provider{
settings: factory.NewScopedProviderSettings(createTestProviderSettings(), "provider_test"),
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := provider.convertLabelSetToEnv(context.Background(), tt.labelSet)
assert.Equal(t, tt.expected, result)
})
}
}

View File

@@ -2,6 +2,8 @@ package envprovider
import (
"context"
"os"
"strings"
"testing"
"github.com/SigNoz/signoz/pkg/config"
@@ -9,7 +11,21 @@ import (
"github.com/stretchr/testify/require"
)
// clearSignozEnv unsets all existing SIGNOZ_* env vars for the duration of the test.
func clearSignozEnv(t *testing.T) {
t.Helper()
for _, kv := range os.Environ() {
if strings.HasPrefix(kv, prefix) {
key := strings.SplitN(kv, "=", 2)[0]
orig, _ := os.LookupEnv(key)
os.Unsetenv(key)
t.Cleanup(func() { os.Setenv(key, orig) })
}
}
}
func TestGetWithStrings(t *testing.T) {
clearSignozEnv(t)
t.Setenv("SIGNOZ_K1_K2", "string")
t.Setenv("SIGNOZ_K3__K4", "string")
t.Setenv("SIGNOZ_K5__K6_K7__K8", "string")
@@ -31,6 +47,7 @@ func TestGetWithStrings(t *testing.T) {
}
func TestGetWithNoPrefix(t *testing.T) {
clearSignozEnv(t)
t.Setenv("K1_K2", "string")
t.Setenv("K3_K4", "string")
expected := map[string]any{}
@@ -43,6 +60,7 @@ func TestGetWithNoPrefix(t *testing.T) {
}
func TestGetWithGoTypes(t *testing.T) {
clearSignozEnv(t)
t.Setenv("SIGNOZ_BOOL", "true")
t.Setenv("SIGNOZ_STRING", "string")
t.Setenv("SIGNOZ_INT", "1")

View File

@@ -65,7 +65,7 @@ func newConfig() factory.Config {
return &Config{
Enabled: false,
Templates: Templates{
Directory: "/root/templates",
Directory: "/root/templates/email",
Format: Format{
Header: Header{
Enabled: false,

View File

@@ -3,14 +3,15 @@ package flagger
import "github.com/SigNoz/signoz/pkg/types/featuretypes"
var (
FeatureUseSpanMetrics = featuretypes.MustNewName("use_span_metrics")
FeatureKafkaSpanEval = featuretypes.MustNewName("kafka_span_eval")
FeatureHideRootUser = featuretypes.MustNewName("hide_root_user")
FeatureGetMetersFromZeus = featuretypes.MustNewName("get_meters_from_zeus")
FeaturePutMetersInZeus = featuretypes.MustNewName("put_meters_in_zeus")
FeatureUseMeterReporter = featuretypes.MustNewName("use_meter_reporter")
FeatureUseJSONBody = featuretypes.MustNewName("use_json_body")
FeatureUseFineGrainedAuthz = featuretypes.MustNewName("use_fine_grained_authz")
FeatureUseSpanMetrics = featuretypes.MustNewName("use_span_metrics")
FeatureKafkaSpanEval = featuretypes.MustNewName("kafka_span_eval")
FeatureHideRootUser = featuretypes.MustNewName("hide_root_user")
FeatureGetMetersFromZeus = featuretypes.MustNewName("get_meters_from_zeus")
FeaturePutMetersInZeus = featuretypes.MustNewName("put_meters_in_zeus")
FeatureUseMeterReporter = featuretypes.MustNewName("use_meter_reporter")
FeatureUseJSONBody = featuretypes.MustNewName("use_json_body")
FeatureUseFineGrainedAuthz = featuretypes.MustNewName("use_fine_grained_authz")
FeatureUseDashboardV2 = featuretypes.MustNewName("use_dashboard_v2")
)
func MustNewRegistry() featuretypes.Registry {
@@ -79,6 +80,14 @@ func MustNewRegistry() featuretypes.Registry {
DefaultVariant: featuretypes.MustNewName("disabled"),
Variants: featuretypes.NewBooleanVariants(),
},
&featuretypes.Feature{
Name: FeatureUseDashboardV2,
Kind: featuretypes.KindBoolean,
Stage: featuretypes.StageExperimental,
Description: "Controls whether dashboard v2 is enabled",
DefaultVariant: featuretypes.MustNewName("disabled"),
Variants: featuretypes.NewBooleanVariants(),
},
)
if err != nil {
panic(err)

View File

@@ -15,12 +15,27 @@ type Config struct {
// Backfill enables sealed-day catch-up from the license creation day.
Backfill bool `mapstructure:"backfill"`
// MaxStartJitter caps the random delay before the first collect after
// Start(). Actual delay is uniform in [0, MaxStartJitter). Negative
// (the default) means "derive from Interval" — see ResolvedMaxStartJitter.
// Set to 0 explicitly to collect immediately on startup (dev / testing).
MaxStartJitter time.Duration `mapstructure:"max_start_jitter"`
// MaxTickJitter is the maximum amount shaved off Interval for each
// subsequent fire. Each cycle = uniform(Interval - MaxTickJitter, Interval].
// Negative (the default) means "derive from Interval" — see
// ResolvedMaxTickJitter. Must be < Interval so two consecutive fires
// cannot collapse to the same instant.
MaxTickJitter time.Duration `mapstructure:"max_tick_jitter"`
}
func newConfig() factory.Config {
return Config{
Interval: 6 * time.Hour,
Backfill: true,
Interval: 6 * time.Hour,
Backfill: true,
MaxStartJitter: -1, // Negative sentinels. Resolved at use time unless explicitly set.
MaxTickJitter: -1, // Negative sentinels. Resolved at use time unless explicitly set.
}
}
@@ -33,5 +48,32 @@ func (c Config) Validate() error {
return errors.New(errors.TypeInvalidInput, ErrCodeInvalidInput, "meterreporter::interval must be between 5m and 24h")
}
if c.MaxStartJitter > c.Interval {
return errors.New(errors.TypeInvalidInput, ErrCodeInvalidInput, "meterreporter::max_start_jitter must be between 0 and interval")
}
if c.MaxTickJitter >= c.Interval {
return errors.New(errors.TypeInvalidInput, ErrCodeInvalidInput, "meterreporter::max_tick_jitter must be in [0, interval)")
}
return nil
}
// ResolvedMaxStartJitter returns the configured MaxStartJitter or, if the
// sentinel default is still in place, falls back to Interval so that the
// jitter scales with whatever Interval the user picks.
func (c Config) ResolvedMaxStartJitter() time.Duration {
if c.MaxStartJitter < 0 {
return c.Interval
}
return c.MaxStartJitter
}
// ResolvedMaxTickJitter returns the configured MaxTickJitter or, if the
// sentinel default is still in place, falls back to Interval / 10.
func (c Config) ResolvedMaxTickJitter() time.Duration {
if c.MaxTickJitter < 0 {
return c.Interval / 10
}
return c.MaxTickJitter
}

View File

@@ -1770,6 +1770,15 @@ func (aH *APIHandler) getFeatureFlags(w http.ResponseWriter, r *http.Request) {
Route: "",
})
useDashboardV2 := aH.Signoz.Flagger.BooleanOrEmpty(r.Context(), flagger.FeatureUseDashboardV2, evalCtx)
featureSet = append(featureSet, &licensetypes.Feature{
Name: valuer.NewString(flagger.FeatureUseDashboardV2.String()),
Active: useDashboardV2,
Usage: 0,
UsageLimit: -1,
Route: "",
})
if constants.IsDotMetricsEnabled {
for idx, feature := range featureSet {
if feature.Name == licensetypes.DotMetricsEnabled {

View File

@@ -352,13 +352,13 @@ func (r *ThresholdRule) Eval(ctx context.Context, ts time.Time) (int, error) {
link := r.prepareLinksToTraces(ctx, ts, smpl.Metric)
if link != "" && r.hostFromSource() != "" {
r.logger.InfoContext(ctx, "adding traces link to annotations", slog.String("annotation.link", fmt.Sprintf("%s/traces-explorer?%s", r.hostFromSource(), link)))
annotations = append(annotations, ruletypes.Label{Name: "related_traces", Value: fmt.Sprintf("%s/traces-explorer?%s", r.hostFromSource(), link)})
annotations = append(annotations, ruletypes.Label{Name: ruletypes.AnnotationRelatedTraces, Value: fmt.Sprintf("%s/traces-explorer?%s", r.hostFromSource(), link)})
}
case ruletypes.AlertTypeLogs:
link := r.prepareLinksToLogs(ctx, ts, smpl.Metric)
if link != "" && r.hostFromSource() != "" {
r.logger.InfoContext(ctx, "adding logs link to annotations", slog.String("annotation.link", fmt.Sprintf("%s/logs/logs-explorer?%s", r.hostFromSource(), link)))
annotations = append(annotations, ruletypes.Label{Name: "related_logs", Value: fmt.Sprintf("%s/logs/logs-explorer?%s", r.hostFromSource(), link)})
annotations = append(annotations, ruletypes.Label{Name: ruletypes.AnnotationRelatedLogs, Value: fmt.Sprintf("%s/logs/logs-explorer?%s", r.hostFromSource(), link)})
}
}

View File

@@ -869,7 +869,7 @@ func TestThresholdRuleTracesLink(t *testing.T) {
assert.Equal(t, c.expectAlerts, alertsFound, "case %d", idx)
for _, item := range rule.Active {
for name, value := range item.Annotations.Map() {
if name == "related_traces" {
if name == ruletypes.AnnotationRelatedTraces {
assert.NotEmpty(t, value, "case %d", idx)
assert.Contains(t, value, "GET")
}
@@ -986,7 +986,7 @@ func TestThresholdRuleLogsLink(t *testing.T) {
assert.Equal(t, c.expectAlerts, alertsFound, "case %d", idx)
for _, item := range rule.Active {
for name, value := range item.Annotations.Map() {
if name == "related_logs" {
if name == ruletypes.AnnotationRelatedLogs {
assert.NotEmpty(t, value, "case %d", idx)
assert.Contains(t, value, "testcontainer")
}

View File

@@ -206,6 +206,7 @@ func NewSQLMigrationProviderFactories(
sqlmigration.NewAddIntegrationDashboardFactory(sqlstore, sqlschema),
sqlmigration.NewAddSourceToDashboardFactory(sqlstore, sqlschema),
sqlmigration.NewMigrateCloudIntegrationDashboardsFactory(sqlstore),
sqlmigration.NewAddScopeToPlannedMaintenanceFactory(sqlstore, sqlschema),
)
}

View File

@@ -0,0 +1,70 @@
package sqlmigration
import (
"context"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/sqlschema"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/uptrace/bun"
"github.com/uptrace/bun/migrate"
)
type addScopeToPlannedMaintenance struct {
sqlstore sqlstore.SQLStore
sqlschema sqlschema.SQLSchema
}
func NewAddScopeToPlannedMaintenanceFactory(sqlstore sqlstore.SQLStore, sqlschema sqlschema.SQLSchema) factory.ProviderFactory[SQLMigration, Config] {
return factory.NewProviderFactory(
factory.MustNewName("add_scope_to_planned"),
func(ctx context.Context, ps factory.ProviderSettings, c Config) (SQLMigration, error) {
return &addScopeToPlannedMaintenance{
sqlstore: sqlstore,
sqlschema: sqlschema,
}, nil
},
)
}
func (migration *addScopeToPlannedMaintenance) Register(migrations *migrate.Migrations) error {
if err := migrations.Register(migration.Up, migration.Down); err != nil {
return err
}
return nil
}
func (migration *addScopeToPlannedMaintenance) Up(ctx context.Context, db *bun.DB) error {
tx, err := db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer func() {
_ = tx.Rollback()
}()
table, _, err := migration.sqlschema.GetTable(ctx, "planned_maintenance")
if err != nil {
return err
}
column := &sqlschema.Column{
Name: sqlschema.ColumnName("scope"),
DataType: sqlschema.DataTypeText,
Nullable: true,
}
sqls := migration.sqlschema.Operator().AddColumn(table, nil, column, nil)
for _, sql := range sqls {
if _, err := tx.ExecContext(ctx, string(sql)); err != nil {
return err
}
}
return tx.Commit()
}
func (migration *addScopeToPlannedMaintenance) Down(ctx context.Context, db *bun.DB) error {
return nil
}

View File

@@ -0,0 +1,88 @@
package alertmanagertypes
import (
"strings"
"github.com/expr-lang/expr"
"github.com/prometheus/common/model"
"github.com/SigNoz/signoz/pkg/errors"
)
var ErrCodeInvalidScopeExpression = errors.MustNewCode("invalid_scope_expression")
// ConvertLabelSetToEnv converts a label set into a map suitable for use as an
// expr environment. Dotted keys (e.g. "kubernetes.node") are expanded into
// nested maps so that expr can resolve them without panicking. When a dotted
// path conflicts with a plain key, the nested structure takes precedence.
//
// The second return value reports whether such a prefix conflict was detected
// (a plain key collided with a nested map, or a nested path overwrote a plain
// leaf).
func ConvertLabelSetToEnv(lset model.LabelSet) (map[string]any, bool) {
env := map[string]any{}
conflict := false
for lk, lv := range lset {
key := strings.TrimSpace(string(lk))
value := string(lv)
if strings.Contains(key, ".") {
parts := strings.Split(key, ".")
current := env
for i, raw := range parts {
part := strings.TrimSpace(raw)
if i == len(parts)-1 {
// Last segment: if a nested map already exists here, a
// deeper path has been processed first — keep it and flag.
if _, isMap := current[part].(map[string]any); isMap {
conflict = true
break
}
current[part] = value
break
}
if nextMap, ok := current[part].(map[string]any); ok {
current = nextMap
} else {
// Intermediate segment hit a plain leaf — overwrite with a
// map so the deeper path can be materialised, and flag.
if _, exists := current[part]; exists {
conflict = true
}
newMap := map[string]any{}
current[part] = newMap
current = newMap
}
}
continue
}
// Plain key collides with an already-built nested map — keep the map
// (nested wins) and flag.
if _, isMap := env[key].(map[string]any); isMap {
conflict = true
continue
}
env[key] = value
}
return env, conflict
}
// EvalScopeExpression compiles and runs the expression against the provided
// labels. It returns (result, error). Callers should log the error and
// decide how to handle a failed evaluation (the maintenance muter treats a
// failure as "don't skip" so alerts pass through).
func EvalScopeExpression(expression string, lset model.LabelSet) (bool, error) {
env, _ := ConvertLabelSetToEnv(lset)
program, err := expr.Compile(expression, expr.Env(env), expr.AllowUndefinedVariables())
if err != nil {
return false, errors.Wrapf(err, errors.TypeInvalidInput, ErrCodeInvalidScopeExpression, "compile scope expression %q", expression)
}
output, err := expr.Run(program, env)
if err != nil {
return false, errors.Wrapf(err, errors.TypeInternal, ErrCodeInvalidScopeExpression, "run scope expression %q", expression)
}
result, ok := output.(bool)
if !ok {
return false, errors.Newf(errors.TypeInvalidInput, ErrCodeInvalidScopeExpression, "scope expression %q returned non-bool value %T (%v)", expression, output, output)
}
return result, nil
}

View File

@@ -0,0 +1,163 @@
package alertmanagertypes
import (
"reflect"
"testing"
"github.com/prometheus/common/model"
)
func TestEvalScopeExpression(t *testing.T) {
cases := []struct {
name string
expression string
lset model.LabelSet
want bool
wantErr bool
}{
{
name: "equality match",
expression: `env = "production"`,
lset: model.LabelSet{"env": "production"},
want: true,
},
{
name: "equality no match",
expression: `env = "production"`,
lset: model.LabelSet{"env": "staging"},
want: false,
},
{
name: "inequality match",
expression: `env != "production"`,
lset: model.LabelSet{"env": "staging"},
want: true,
},
{
name: "AND - both match",
expression: `env = "production" AND service = "api"`,
lset: model.LabelSet{"env": "production", "service": "api"},
want: true,
},
{
name: "AND - partial match",
expression: `env = "production" AND service = "api"`,
lset: model.LabelSet{"env": "production", "service": "worker"},
want: false,
},
{
name: "OR - first matches",
expression: `env = "production" OR env = "staging"`,
lset: model.LabelSet{"env": "production"},
want: true,
},
{
name: "OR - second matches",
expression: `env = "production" OR env = "staging"`,
lset: model.LabelSet{"env": "staging"},
want: true,
},
{
name: "OR - none match",
expression: `env = "production" OR env = "staging"`,
lset: model.LabelSet{"env": "development"},
want: false,
},
{
name: "undefined label returns false",
expression: `env = "production"`,
lset: model.LabelSet{"service": "api"},
want: false,
},
{
name: "in list - present",
expression: `env in ["production", "staging"]`,
lset: model.LabelSet{"env": "production"},
want: true,
},
{
name: "in list - absent",
expression: `env in ["production", "staging"]`,
lset: model.LabelSet{"env": "development"},
want: false,
},
{
name: "invalid expression returns error",
expression: `env =`,
lset: model.LabelSet{"env": "production"},
want: false,
wantErr: true,
},
{
name: "non-bool expression returns error",
expression: `env`,
lset: model.LabelSet{"env": "production"},
want: false,
wantErr: true,
},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
got, err := EvalScopeExpression(c.expression, c.lset)
if (err != nil) != c.wantErr {
t.Errorf("EvalScopeExpression(%q, %v) error = %v, wantErr %v", c.expression, c.lset, err, c.wantErr)
}
if got != c.want {
t.Errorf("EvalScopeExpression(%q, %v) = %v, want %v", c.expression, c.lset, got, c.want)
}
})
}
}
func TestConvertLabelSetToEnv(t *testing.T) {
cases := []struct {
name string
lset model.LabelSet
expected map[string]interface{}
wantConflict bool
}{
{
name: "simple keys",
lset: model.LabelSet{"key1": "value1", "key2": "value2"},
expected: map[string]interface{}{"key1": "value1", "key2": "value2"},
},
{
name: "dotted keys become nested maps",
lset: model.LabelSet{"foo.bar": "value1", "foo.baz": "value2"},
expected: map[string]interface{}{
"foo": map[string]interface{}{"bar": "value1", "baz": "value2"},
},
},
{
name: "deeper dotted key wins over shallow dotted key",
lset: model.LabelSet{"foo.bar.baz": "deep", "foo.bar": "shallow"},
expected: map[string]interface{}{
"foo": map[string]interface{}{
"bar": map[string]interface{}{"baz": "deep"},
},
},
wantConflict: true,
},
{
name: "nested structure wins over plain key",
lset: model.LabelSet{"foo.bar": "value", "foo": "ignored"},
expected: map[string]interface{}{
"foo": map[string]interface{}{"bar": "value"},
},
wantConflict: true,
},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
got, gotConflict := ConvertLabelSetToEnv(c.lset)
if !reflect.DeepEqual(got, c.expected) {
t.Errorf("ConvertLabelSetToEnv() map = %v, want %v", got, c.expected)
}
if gotConflict != c.wantConflict {
t.Errorf("ConvertLabelSetToEnv() conflict = %v, want %v", gotConflict, c.wantConflict)
}
})
}
}

View File

@@ -5,14 +5,19 @@ import (
"encoding/json"
"time"
"github.com/expr-lang/expr"
"github.com/prometheus/common/model"
"github.com/uptrace/bun"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/uptrace/bun"
)
var ErrCodeInvalidPlannedMaintenancePayload = errors.MustNewCode("invalid_planned_maintenance_payload")
const scopeDocUrl = "https://signoz.io/docs/alerts-management/planned-maintenance/#scoping-with-label-expressions"
type MaintenanceStatus struct {
valuer.String
}
@@ -58,6 +63,7 @@ type StorablePlannedMaintenance struct {
Description string `bun:"description,type:text"`
Schedule *Schedule `bun:"schedule,type:text,notnull"`
OrgID string `bun:"org_id,type:text"`
Scope string `bun:"scope,type:text"`
}
type PlannedMaintenance struct {
@@ -66,6 +72,7 @@ type PlannedMaintenance struct {
Description string `json:"description"`
Schedule *Schedule `json:"schedule" required:"true"`
RuleIDs []string `json:"alertIds"`
Scope string `json:"scope,omitempty"`
CreatedAt time.Time `json:"createdAt"`
CreatedBy string `json:"createdBy"`
UpdatedAt time.Time `json:"updatedAt"`
@@ -82,6 +89,7 @@ type PostablePlannedMaintenance struct {
Description string `json:"description"`
Schedule *Schedule `json:"schedule" required:"true"`
AlertIds []string `json:"alertIds"`
Scope string `json:"scope"`
}
func (p *PostablePlannedMaintenance) Validate() error {
@@ -116,6 +124,15 @@ func (p *PostablePlannedMaintenance) Validate() error {
return errors.Newf(errors.TypeInvalidInput, ErrCodeInvalidPlannedMaintenancePayload, "end time cannot be before start time")
}
}
if p.Scope != "" {
if _, err := expr.Compile(p.Scope, expr.AllowUndefinedVariables(), expr.AsBool()); err != nil {
err := errors.Newf(
errors.TypeInvalidInput, ErrCodeInvalidPlannedMaintenancePayload,
"invalid scope: %s", err.Error(),
)
return err.WithUrl(scopeDocUrl)
}
}
return nil
}
@@ -151,7 +168,7 @@ func (m *PlannedMaintenance) HasScheduleRecurrenceBoundsMismatch() bool {
(recurrence.EndTime != nil && !recurrence.EndTime.Equal(m.Schedule.EndTime))
}
func (m *PlannedMaintenance) ShouldSkip(ruleID string, now time.Time) bool {
func (m *PlannedMaintenance) ShouldSkip(ruleID string, now time.Time, lset model.LabelSet) (bool, error) {
// Check if the alert ID is in the maintenance window
found := false
if len(m.RuleIDs) > 0 {
@@ -168,9 +185,27 @@ func (m *PlannedMaintenance) ShouldSkip(ruleID string, now time.Time) bool {
}
if !found {
return false
return false, nil
}
if !m.IsActive(now) {
return false, nil
}
if m.Scope != "" {
result, err := EvalScopeExpression(m.Scope, lset)
if err != nil {
return false, err
}
if !result {
return false, nil
}
}
return true, nil
}
// IsActive reports whether [now] falls inside the maintenance window's schedule.
func (m *PlannedMaintenance) IsActive(now time.Time) bool {
// If alert is found, we check if it should be skipped based on the schedule
loc, err := time.LoadLocation(m.Schedule.Timezone)
if err != nil {
@@ -301,14 +336,6 @@ func (m *PlannedMaintenance) checkMonthly(currentTime time.Time, rec *Recurrence
return currentTime.Sub(candidate) <= rec.Duration.Duration()
}
func (m *PlannedMaintenance) IsActive(now time.Time) bool {
ruleID := "maintenance"
if len(m.RuleIDs) > 0 {
ruleID = (m.RuleIDs)[0]
}
return m.ShouldSkip(ruleID, now)
}
func (m *PlannedMaintenance) IsUpcoming() bool {
loc, err := time.LoadLocation(m.Schedule.Timezone)
if err != nil {
@@ -389,6 +416,7 @@ func (m PlannedMaintenance) MarshalJSON() ([]byte, error) {
Description string `json:"description" db:"description"`
Schedule *Schedule `json:"schedule" db:"schedule"`
AlertIds []string `json:"alertIds" db:"alert_ids"`
Scope string `json:"scope,omitempty" db:"scope"`
CreatedAt time.Time `json:"createdAt" db:"created_at"`
CreatedBy string `json:"createdBy" db:"created_by"`
UpdatedAt time.Time `json:"updatedAt" db:"updated_at"`
@@ -401,6 +429,7 @@ func (m PlannedMaintenance) MarshalJSON() ([]byte, error) {
Description: m.Description,
Schedule: m.Schedule,
AlertIds: m.RuleIDs,
Scope: m.Scope,
CreatedAt: m.CreatedAt,
CreatedBy: m.CreatedBy,
UpdatedAt: m.UpdatedAt,
@@ -424,6 +453,7 @@ func (m *PlannedMaintenanceWithRules) ToPlannedMaintenance() *PlannedMaintenance
Description: m.Description,
Schedule: m.Schedule,
RuleIDs: ruleIDs,
Scope: m.Scope,
CreatedAt: m.CreatedAt,
UpdatedAt: m.UpdatedAt,
CreatedBy: m.CreatedBy,

View File

@@ -5,6 +5,7 @@ import (
"time"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/prometheus/common/model"
)
// Helper function to create a time pointer.
@@ -668,9 +669,193 @@ func TestShouldSkipMaintenance(t *testing.T) {
}
for idx, c := range cases {
result := c.maintenance.ShouldSkip(c.name, c.ts)
result, err := c.maintenance.ShouldSkip(c.name, c.ts, model.LabelSet{})
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if result != c.skip {
t.Errorf("skip %v, got %v, case:%d - %s", c.skip, result, idx, c.name)
}
}
}
func TestShouldSkip_Scope(t *testing.T) {
activeSchedule := func() *Schedule {
return &Schedule{
Timezone: "UTC",
StartTime: time.Now().UTC().Add(-time.Hour),
EndTime: time.Now().UTC().Add(time.Hour),
}
}
now := time.Now().UTC()
cases := []struct {
name string
maintenance *PlannedMaintenance
ruleID string
ts time.Time
lset model.LabelSet
skip bool
}{
{
name: "empty scope - no label filtering applied",
maintenance: &PlannedMaintenance{Schedule: activeSchedule()},
ruleID: "rule-1",
ts: now,
lset: model.LabelSet{"env": "production"},
skip: true,
},
{
name: "scope matches labels",
maintenance: &PlannedMaintenance{Schedule: activeSchedule(), Scope: `env = "production"`},
ruleID: "rule-1",
ts: now,
lset: model.LabelSet{"env": "production"},
skip: true,
},
{
name: "scope does not match labels",
maintenance: &PlannedMaintenance{Schedule: activeSchedule(), Scope: `env = "production"`},
ruleID: "rule-1",
ts: now,
lset: model.LabelSet{"env": "staging"},
skip: false,
},
{
name: "AND expression - both conditions match",
maintenance: &PlannedMaintenance{Schedule: activeSchedule(), Scope: `env = "production" AND service = "api"`},
ruleID: "rule-1",
ts: now,
lset: model.LabelSet{"env": "production", "service": "api"},
skip: true,
},
{
name: "AND expression - one condition does not match",
maintenance: &PlannedMaintenance{Schedule: activeSchedule(), Scope: `env = "production" AND service = "api"`},
ruleID: "rule-1",
ts: now,
lset: model.LabelSet{"env": "production", "service": "worker"},
skip: false,
},
{
name: "OR expression - first alternative matches",
maintenance: &PlannedMaintenance{Schedule: activeSchedule(), Scope: `env = "production" OR env = "staging"`},
ruleID: "rule-1",
ts: now,
lset: model.LabelSet{"env": "production"},
skip: true,
},
{
name: "OR expression - second alternative matches",
maintenance: &PlannedMaintenance{Schedule: activeSchedule(), Scope: `env = "production" OR env = "staging"`},
ruleID: "rule-1",
ts: now,
lset: model.LabelSet{"env": "staging"},
skip: true,
},
{
name: "OR expression - neither alternative matches",
maintenance: &PlannedMaintenance{Schedule: activeSchedule(), Scope: `env = "production" OR env = "staging"`},
ruleID: "rule-1",
ts: now,
lset: model.LabelSet{"env": "development"},
skip: false,
},
{
name: "scope references label absent from lset",
maintenance: &PlannedMaintenance{Schedule: activeSchedule(), Scope: `env = "production"`},
ruleID: "rule-1",
ts: now,
lset: model.LabelSet{"service": "api"},
skip: false,
},
{
name: "in expression - value is in list",
maintenance: &PlannedMaintenance{Schedule: activeSchedule(), Scope: `env in ["production", "staging"]`},
ruleID: "rule-1",
ts: now,
lset: model.LabelSet{"env": "staging"},
skip: true,
},
{
name: "in expression - value not in list",
maintenance: &PlannedMaintenance{Schedule: activeSchedule(), Scope: `env in ["production", "staging"]`},
ruleID: "rule-1",
ts: now,
lset: model.LabelSet{"env": "development"},
skip: false,
},
{
name: "ruleID in list and scope matches - should skip",
maintenance: &PlannedMaintenance{Schedule: activeSchedule(), RuleIDs: []string{"rule-1", "rule-2"}, Scope: `env = "production"`},
ruleID: "rule-1",
ts: now,
lset: model.LabelSet{"env": "production"},
skip: true,
},
{
name: "ruleID not in list and scope matches - ruleID gate prevents skip",
maintenance: &PlannedMaintenance{Schedule: activeSchedule(), RuleIDs: []string{"rule-2"}, Scope: `env = "production"`},
ruleID: "rule-1",
ts: now,
lset: model.LabelSet{"env": "production"},
skip: false,
},
{
name: "ruleID in list but scope does not match - should not skip",
maintenance: &PlannedMaintenance{Schedule: activeSchedule(), RuleIDs: []string{"rule-1"}, Scope: `env = "production"`},
ruleID: "rule-1",
ts: now,
lset: model.LabelSet{"env": "staging"},
skip: false,
},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
got, err := c.maintenance.ShouldSkip(c.ruleID, c.ts, c.lset)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if got != c.skip {
t.Errorf("ShouldSkip() = %v, want %v", got, c.skip)
}
})
}
}
func TestPostablePlannedMaintenance_ValidateScope(t *testing.T) {
validSchedule := &Schedule{
Timezone: "UTC",
StartTime: time.Now().UTC(),
EndTime: time.Now().UTC().Add(time.Hour),
}
cases := []struct {
name string
scope string
wantErr bool
}{
{name: "empty scope", scope: "", wantErr: false},
{name: "simple equality", scope: `env = "production"`, wantErr: false},
{name: "AND expression", scope: `env = "production" AND service = "api"`, wantErr: false},
{name: "OR expression", scope: `env = "production" OR env = "staging"`, wantErr: false},
{name: "in expression", scope: `env in ["production", "staging"]`, wantErr: false},
{name: "incomplete expression", scope: `env =`, wantErr: true},
{name: "non-bool expression", scope: `"just a string"`, wantErr: true},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
p := &PostablePlannedMaintenance{
Name: "test",
Schedule: validSchedule,
Scope: c.scope,
}
err := p.Validate()
if (err != nil) != c.wantErr {
t.Errorf("Validate() error = %v, wantErr %v", err, c.wantErr)
}
})
}
}

View File

@@ -0,0 +1,20 @@
package alertmanagertypes
import (
"context"
"log/slog"
"github.com/prometheus/alertmanager/notify"
"github.com/prometheus/alertmanager/template"
"github.com/prometheus/alertmanager/types"
)
// Templater expands user-authored title and body templates against a group
// of alerts. Implemented by pkg/alertmanager/alertmanagertemplate.
type Templater interface {
Expand(ctx context.Context, req ExpandRequest, alerts []*types.Alert) (*ExpandResult, error)
}
// ReceiverIntegrationsFunc constructs the notify.Integration list for a
// configured receiver.
type ReceiverIntegrationsFunc = func(nc Receiver, tmpl *template.Template, logger *slog.Logger, templater Templater) ([]notify.Integration, error)

View File

@@ -19,8 +19,7 @@ import (
type (
// Receiver is the type for the receiver configuration.
Receiver = config.Receiver
ReceiverIntegrationsFunc = func(nc Receiver, tmpl *template.Template, logger *slog.Logger) ([]notify.Integration, error)
Receiver = config.Receiver
)
// Creates a new receiver from a string. The input is initialized with the default values from the upstream alertmanager.
@@ -51,7 +50,7 @@ func NewReceiver(input string) (Receiver, error) {
return receiverWithDefaults, nil
}
func TestReceiver(ctx context.Context, receiver Receiver, receiverIntegrationsFunc ReceiverIntegrationsFunc, config *Config, tmpl *template.Template, logger *slog.Logger, lSet model.LabelSet, alert ...*Alert) error {
func TestReceiver(ctx context.Context, receiver Receiver, receiverIntegrationsFunc ReceiverIntegrationsFunc, config *Config, tmpl *template.Template, logger *slog.Logger, templater Templater, lSet model.LabelSet, alert ...*Alert) error {
ctx = notify.WithGroupKey(ctx, fmt.Sprintf("%s-%s-%d", receiver.Name, lSet.Fingerprint(), time.Now().Unix()))
ctx = notify.WithGroupLabels(ctx, lSet)
ctx = notify.WithReceiverName(ctx, receiver.Name)
@@ -73,7 +72,7 @@ func TestReceiver(ctx context.Context, receiver Receiver, receiverIntegrationsFu
return err
}
integrations, err := receiverIntegrationsFunc(receiver, tmpl, logger)
integrations, err := receiverIntegrationsFunc(receiver, tmpl, logger, templater)
if err != nil {
return err
}

View File

@@ -108,6 +108,7 @@ func (s *Schedule) UnmarshalJSON(data []byte) error {
if err != nil {
return err
}
// TODO(jatinderjit): if endTime.IsZero() then we should not set the endTime
s.EndTime = time.Date(endTime.Year(), endTime.Month(), endTime.Day(), endTime.Hour(), endTime.Minute(), endTime.Second(), endTime.Nanosecond(), loc)
}

View File

@@ -77,6 +77,28 @@ func (c CompareOperator) Normalize() CompareOperator {
}
}
// Literal returns the canonical literal (string) form of the operator.
func (c CompareOperator) Literal() string {
switch c.Normalize() {
case ValueIsAbove:
return ValueIsAboveLiteral.StringValue()
case ValueIsBelow:
return ValueIsBelowLiteral.StringValue()
case ValueIsEq:
return ValueIsEqLiteral.StringValue()
case ValueIsNotEq:
return ValueIsNotEqLiteral.StringValue()
case ValueAboveOrEq:
return ValueAboveOrEqLiteral.StringValue()
case ValueBelowOrEq:
return ValueBelowOrEqLiteral.StringValue()
case ValueOutsideBounds:
return ValueOutsideBoundsLiteral.StringValue()
default:
return c.StringValue()
}
}
func (c CompareOperator) Validate() error {
switch c {
case ValueIsAbove,

View File

@@ -56,6 +56,24 @@ func (m MatchType) Normalize() MatchType {
}
}
// Literal returns the canonical literal (string) form of the match type.
func (m MatchType) Literal() string {
switch m.Normalize() {
case AtleastOnce:
return AtleastOnceLiteral.StringValue()
case AllTheTimes:
return AllTheTimesLiteral.StringValue()
case OnAverage:
return OnAverageLiteral.StringValue()
case InTotal:
return InTotalLiteral.StringValue()
case Last:
return LastLiteral.StringValue()
default:
return m.StringValue()
}
}
func (m MatchType) Validate() error {
switch m {
case

View File

@@ -24,6 +24,10 @@ type Sample struct {
RecoveryTarget *float64
TargetUnit string
// CompareOperator and MatchType carry the threshold evaluation context
CompareOperator CompareOperator
MatchType MatchType
}
func (s Sample) String() string {

View File

@@ -188,6 +188,8 @@ func (r BasicRuleThresholds) Eval(s *qbtypes.TimeSeries, unit string, evalData E
smpl.RecoveryTarget = threshold.RecoveryTarget
}
smpl.TargetUnit = threshold.TargetUnit
smpl.CompareOperator = threshold.CompareOperator
smpl.MatchType = threshold.MatchType
resultVector = append(resultVector, smpl)
continue
} else if evalData.SendUnmatched {
@@ -197,10 +199,12 @@ func (r BasicRuleThresholds) Eval(s *qbtypes.TimeSeries, unit string, evalData E
}
// prepare the sample with the first point of the series
smpl := Sample{
Point: Point{T: series.Values[0].Timestamp, V: series.Values[0].Value},
Metric: PrepareSampleLabelsForRule(series.Labels, threshold.Name),
Target: *threshold.TargetValue,
TargetUnit: threshold.TargetUnit,
Point: Point{T: series.Values[0].Timestamp, V: series.Values[0].Value},
Metric: PrepareSampleLabelsForRule(series.Labels, threshold.Name),
Target: *threshold.TargetValue,
TargetUnit: threshold.TargetUnit,
CompareOperator: threshold.CompareOperator,
MatchType: threshold.MatchType,
}
if threshold.RecoveryTarget != nil {
smpl.RecoveryTarget = threshold.RecoveryTarget
@@ -222,6 +226,8 @@ func (r BasicRuleThresholds) Eval(s *qbtypes.TimeSeries, unit string, evalData E
smpl.Target = *threshold.TargetValue
smpl.RecoveryTarget = threshold.RecoveryTarget
smpl.TargetUnit = threshold.TargetUnit
smpl.CompareOperator = threshold.CompareOperator
smpl.MatchType = threshold.MatchType
// IsRecovering to notify that metrics is in recovery stage
smpl.IsRecovering = true
resultVector = append(resultVector, smpl)

View File

@@ -0,0 +1,122 @@
{{ define "email.signoz.html" }}
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<title>{{.Title}}</title>
<style>
code {
background: #f0f0f0;
padding: 2px 6px;
border-radius: 3px;
font-family: 'SFMono-Regular', Consolas, 'Liberation Mono', Menlo, monospace;
font-size: 13px;
}
pre {
background: #f0f0f0;
padding: 12px 16px;
border-radius: 6px;
font-size: 13px;
overflow-x: auto;
white-space: pre;
}
pre code {
background: none;
padding: 0;
border-radius: 0;
font-size: inherit;
}
table:not([role="presentation"]) {
width: 100%;
border-collapse: collapse;
font-size: 14px;
}
table:not([role="presentation"]) th {
font-weight: 600;
text-align: left;
padding: 8px 12px;
border-bottom: 2px solid #d0d0d0;
}
table:not([role="presentation"]) td {
padding: 8px 12px;
border-bottom: 1px solid #e8e8e8;
}
table:not([role="presentation"]) tr:last-child td {
border-bottom: none;
}
</style>
</head>
<body style="margin:0;padding:0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,'Helvetica Neue',Arial,sans-serif;line-height:1.6;color:#333;background:#fff">
<table role="presentation" width="100%" cellspacing="0" cellpadding="0" border="0" style="background:#fff">
<tr>
<td align="center" style="padding:0">
<table role="presentation" width="600" cellspacing="0" cellpadding="0" border="0" style="max-width:600px;width:100%;border:1px solid #e2e2e2;border-radius:12px;overflow:hidden">
<tr>
<td align="center" style="padding:20px 20px 12px">
<h2 style="margin:0 0 8px;font-size:20px;color:#333">{{.Title}}</h2>
<p style="margin:0;font-size:14px;color:#666">
Status: <strong>{{.Alert.Status}}</strong>
{{if .Alert.TotalFiring}} | Firing: <strong style="color:#e53e3e">{{.Alert.TotalFiring}}</strong>{{end}}
{{if .Alert.TotalResolved}} | Resolved: <strong style="color:#38a169">{{.Alert.TotalResolved}}</strong>{{end}}
</p>
</td>
</tr>
<tr>
<td style="padding:0 20px">
<table role="presentation" width="100%" cellspacing="0" cellpadding="0" border="0">
<tr><td style="border-top:1px solid #e2e2e2;font-size:0;line-height:0" height="1">&nbsp;</td></tr>
</table>
</td>
</tr>
{{range .Bodies}}
<tr>
<td style="padding:8px 20px">
<table role="presentation" width="100%" cellspacing="0" cellpadding="0" border="0">
<tr>
<td style="padding:16px;background:#fafafa;border:1px solid #e8e8e8;border-radius:6px">
{{.}}
</td>
</tr>
</table>
</td>
</tr>
{{end}}
{{if .NotificationTemplateData.ExternalURL}}
<tr>
<td style="padding:16px 20px">
<table role="presentation" width="100%" cellspacing="0" cellpadding="0" border="0">
<tr>
<td align="center">
<a href="{{.NotificationTemplateData.ExternalURL}}" target="_blank" style="display:inline-block;padding:12px 32px;font-size:14px;font-weight:600;color:#fff;background:#4E74F8;text-decoration:none;border-radius:4px">
View in SigNoz
</a>
</td>
</tr>
</table>
</td>
</tr>
{{end}}
<tr>
<td align="center" style="padding:8px 16px 16px">
<p style="margin:0;font-size:12px;color:#999;line-height:1.5">
Sent by SigNoz AlertManager
</p>
</td>
</tr>
</table>
</td>
</tr>
</table>
</body>
</html>
{{ end }}