SigNoz/signoz
1
0
Fork 0
mirror of https://github.com/SigNoz/signoz.git synced 2026-09-22 19:30:43 +01:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity
7,124 Commits 656 Branches 739 Tags
81afa641a2ebb9376d3aba2cecdfd49ba3efcfac
Commit Graph

5 Commits

Author SHA1 Message Date
Nikhil Soni
a6346183e8 feat(traces-qb): read span attributes from the JSON column (evolution-gated) (#12715) 
#### Description

Enables the v5 traces query builder to read span attributes from the
native `attributes` `JSON(max_dynamic_paths=0)` column, alongside the
legacy `attributes_string/number/bool` maps, which will get deprecated
over time once the dual ingestion can be stopped.

- **Evolution-gated rollout.** A key resolves to the JSON column only
once its column-evolution set names `attributes`; with no such entry it
resolves to the Map column exactly as before.
- **Negative operators** - In currently live system, records with
missing keys for number/bool types are included in the negative operator
queries while string data type is not included. So we are preserving the
same be
- Data-type-unspecified attribute keys keep the existing
`CandidateKeys`/synthesis path (no branch-flip of the most common query
shape) - will be fixed in
https://github.com/SigNoz/engineering-pod/issues/6018

Part of https://github.com/SigNoz/engineering-pod/issues/5878 

#### Additional Information

First of a stacked series. Using json in span listing and other module
like waterfall, flamegraph will be follow up changes tracked in
https://github.com/SigNoz/engineering-pod/issues/5967

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-09-10 14:38:08 +00:00
Vinicius Lourenço
a8c04cb563 test(alerts): add e2e for alerts (#12349) 
## Pull Request

---

### 📄 Summary
> Why does this change exist?  
> What problem does it solve, and why is this the right approach?

This adds a bunch of E2E tests for alerts, to test v1/v2 create and
edit, and also tests for alert history.

This started as tests only for history but decided to just add tests for
everything, while creating them, I found two bugs inside alerts, so they
already helping us before even landing :)

The changes in the UI are only to add testIds, no change in logic (and
no fix for the incidents)

| Scope | Before (`main`) | After (this branch) | Delta |
|---|---:|---:|---:|
| Alerts E2E tests | 2 | 191 | **+189** |
| Alerts E2E spec files | 1 | 31 | +30 |
| Whole E2E suite | 141 | 330 | **+189** |


#### Alerts page shell (7)

| File | Test | Status |
|---|---|---|
| `page.spec.ts` | AL-01 all four top-level tabs render |  |
| `page.spec.ts` | AL-02 default tab is Alert Rules |  |
| `page.spec.ts` | AL-03 tab switch writes ?tab= and clears subTab |  |
| `page.spec.ts` | AL-04 Configuration deep-link |  |
| `page.spec.ts` | AL-05 Triggered Alerts tab smoke |  |
| `page.spec.ts` | AL-06 Notification Channels tab smoke |  |
| `page.spec.ts` | AL-07 tab state survives reload |  |

#### Alert rules list (19)

| File | Test | Status |
|---|---|---|
| `list/columns.spec.ts` | LR-01 renders all default columns (Status,
Alert Name, Severity, Labels, Actions) | |
| `list/columns.spec.ts` | LR-02 shows empty state when no rules exist |
skipped |
| `list/columns.spec.ts` | LR-10 column selector hides and shows a
column | |
| `list/navigation.spec.ts` | LR-11 row click opens the overview page |
|
| `list/navigation.spec.ts` | LR-12 ctrl/cmd-click opens the overview in
a new tab | |
| `list/navigation.spec.ts` | LR-13 actions menu Edit and Edit in New
Tab navigate correctly | |
| `list/navigation.spec.ts` | LR-17 New Alert button navigates to alert
creation | |
| `list/navigation.spec.ts` | LR-18 shows ErrorEmptyState when list
fails to load | skipped |
| `list/pagination-sort.spec.ts` | LR-07 navigates between pages |  |
| `list/pagination-sort.spec.ts` | LR-08 changes page size |  |
| `list/pagination-sort.spec.ts` | LR-09 sorts by column header click |
|
| `list/row-actions.spec.ts` | LR-14 Disable then Enable toggles the
rule state | |
| `list/row-actions.spec.ts` | LR-15 Clone creates a copy and shows
success toast | |
| `list/row-actions.spec.ts` | LR-16 Delete removes the rule and shows
success toast | |
| `list/search.spec.ts` | LR-03 filters by name |  |
| `list/search.spec.ts` | LR-04 filters by severity and by label |  |
| `list/search.spec.ts` | LR-05 shows no-results state with clear button
| |
| `list/search.spec.ts` | LR-06 resets pagination when searching |  |
| `list/search.spec.ts` | LR-19 state and severity filters intersect,
they do not union | |

#### Create alert (52)

| File | Test | Status |
|---|---|---|
| `create/edge.spec.ts` | CE-04 a server-side rejection opens the error
modal and keeps the draft | |
| `create/edge.spec.ts` | CE-07 none of the four builder mounts logs a
console error | |
| `create/edge.spec.ts` | CE-09 the v2 Discard button is clickable |
skipped |
| `create/prefill.spec.ts` | CD-01 a compositeQuery alone selects the
alert type | |
| `create/prefill.spec.ts` | CD-02 thresholds prefill from JSON, and a
malformed value falls back | |
| `create/prefill.spec.ts` | CD-03 matchType and compareOp aliases
normalise to the enum | |
| `create/prefill.spec.ts` | CD-04 ruleName and yAxisUnit apply once and
never stomp an edit | |
| `create/prefill.spec.ts` | CD-05 evaluationWindowPreset=meter switches
to the cumulative daily window | |
| `create/prefill.spec.ts` | CD-06 URL prefill is ignored in edit mode |
|
| `create/shell.spec.ts` | CS-01 bare /alerts/new lists exactly the
expected alert-type cards | |
| `create/shell.spec.ts` | CS-02 picking a card writes both params and
mounts the v2 builder | |
| `create/shell.spec.ts` | CS-03 the anomaly card rewrites the rule
type, not the alert type | conditional |
| `create/shell.spec.ts` | CS-04 modifier-clicking a card opens the
builder in a new tab | |
| `create/shell.spec.ts` | CS-05 breadcrumb gains a third crumb after a
type is picked | |
| `create/shell.spec.ts` | CS-06 create renders inside the Alert Rules
tab and leaving drops subTab/search | |
| `create/shell.spec.ts` | CS-07 showClassicCreateAlertsPage=true
renders the v1 form instead | |
| `create/shell.spec.ts` | CS-08 Switch to Classic Experience replaces
history, so Back does not return to v2 | |
| `create/v1.spec.ts` | CV1-01 the classic form renders its steps and
the create-mode labels | |
| `create/v1.spec.ts` | CV1-02 the rendered severity is the default from
the rule, not the select | |
| `create/v1.spec.ts` | CV1-03 one keystroke in the name field is enough
to enable Save | |
| `create/v1.spec.ts` | CV1-04 Save stays disabled until the channel
configuration resolves | |
| `create/v1.spec.ts` | CV1-05 broadcast-to-all saves the rule with the
broadcast flag | skipped |
| `create/v1.spec.ts` | CV1-06 a cleared threshold is coerced to 0, so
the required-threshold branch is dead | |
| `create/v1.spec.ts` | CV1-07 cancelling the confirm dialog does not
save | |
| `create/v1.spec.ts` | CV1-08 the happy path posts the v1 body shape to
the shared endpoint | |
| `create/v1.spec.ts` | CV1-09 CV1-10 description, labels and severity
all land in the payload | |
| `create/v1.spec.ts` | CV1-11 test notification skips the dialog and
reports no matching data | |
| `create/v1.spec.ts` | CV1-12 with no channels the form is a dead end |
|
| `create/v1.spec.ts` | CV1-13 Cancel leaves the form without saving | |
| `create/v1.spec.ts` | CE-05 an empty PromQL expression is rejected
behind the dialog | |
| `create/v1.spec.ts` | CE-06 an empty ClickHouse query is rejected
behind the dialog | |
| `create/v1.spec.ts` | CV1-14 the condition sentence keeps its
selections | |
| `create/v2.spec.ts` | CV2-01 initial state: one critical threshold,
both actions gated | |
| `create/v2.spec.ts` | CV2-02 the save tooltip walks from the name gate
to the channel gate | |
| `create/v2.spec.ts` | CV2-03 clearing a threshold label re-gates the
save | |
| `create/v2.spec.ts` | CV2-04 a label added in the header survives the
save round-trip | |
| `create/v2.spec.ts` | CV2-05 a rejected label key surfaces as a
notification, not an inline message | |
| `create/v2.spec.ts` | CV2-06 CV2-07 the operator and match-type
selects offer the documented options | |
| `create/v2.spec.ts` | CV2-08 the operator is rule-wide: one change
reaches every threshold | |
| `create/v2.spec.ts` | CV2-09 CV2-10 added thresholds take preset
tiers, and the first cannot be removed | |
| `create/v2.spec.ts` | CV2-11 a channel on one threshold is not enough
— the validator loops all of them | |
| `create/v2.spec.ts` | CV2-12 the unit select is disabled while the
query has no y-axis unit | |
| `create/v2.spec.ts` | CV2-13 the recovery threshold control is never
rendered | |
| `create/v2.spec.ts` | CV2-14 CV2-15 the evaluation window and cadence
reach the payload | |
| `create/v2.spec.ts` | CV2-18 with no channels the dropdown offers only
a way to create one | |
| `create/v2.spec.ts` | CV2-19 routing policies unlock the save with
zero channels | |
| `create/v2.spec.ts` | CV2-16 the group-by select is disabled until the
query groups by something | |
| `create/v2.spec.ts` | CV2-17 repeat notifications enable their inputs
and reach the payload | |
| `create/v2.spec.ts` | CV2-20 happy-path save posts the v2 shape and
lands on the list | |
| `create/v2.spec.ts` | CV2-21 test notification reports that a
non-firing rule matched nothing | |
| `create/v2.spec.ts` | CV2-22 discard leaves without posting and resets
the form | |
| `create/v2.spec.ts` | CV2-23 every footer button is disabled while the
save is in flight | |

#### Edit alert (22)

| File | Test | Status |
|---|---|---|
| `edit/edge.spec.ts` | CE-03 an unknown ruleId shows AlertNotFound on
both entry URLs | |
| `edit/edge.spec.ts` | CE-03b /alerts/edit with no ruleId also lands on
AlertNotFound | |
| `edit/v1.spec.ts` | EV1-01 the classic form renders in edit mode
inside the details shell | |
| `edit/v1.spec.ts` | EV1-02 every seeded field prefills the form |  |
| `edit/v1.spec.ts` | EV1-03 preferredChannels decide which channel
control is prefilled | |
| `edit/v1.spec.ts` | EV1-04 the happy-path update PUTs the v1 body and
keeps unrelated params | |
| `edit/v1.spec.ts` | EV1-05 Discard leaves without a PUT and without
changing the rule | |
| `edit/v1.spec.ts` | EV1-06 the header title and the form name field
agree | |
| `edit/v1.spec.ts` | EV1-07 /alerts/edit redirects for a v1 rule
exactly as it does for v2 | |
| `edit/v1.spec.ts` | EV1-08 editing a v1 rule never migrates it to the
v2 schema | |
| `edit/v2.spec.ts` | EV2-01 the v2 editor renders inside the details
shell | |
| `edit/v2.spec.ts` | EV2-02 name and labels prefill from the rule |  |
| `edit/v2.spec.ts` | EV2-03 both thresholds prefill, and the sentence
reads spec[0] | |
| `edit/v2.spec.ts` | EV2-04 the recovery threshold control never
renders | |
| `edit/v2.spec.ts` | EV2-05 the evaluation window prefills, and a
non-preset value collapses to custom | |
| `edit/v2.spec.ts` | EV2-06 repeat notifications prefill from the
seeded renotify block | |
| `edit/v2.spec.ts` | EV2-07 alertOnAbsent prefills the advanced options
| |
| `edit/v2.spec.ts` | EV2-08 the evaluation cadence always reads back in
default mode | |
| `edit/v2.spec.ts` | EV2-09 changing a threshold PUTs the rule and the
change survives a reload | |
| `edit/v2.spec.ts` | EV2-10 the footer save is what persists a rename
made on the Overview tab | |
| `edit/v2.spec.ts` | EV2-11 Discard leaves without a PUT and without
touching the rule | |
| `edit/v2.spec.ts` | EV2-12 /alerts/edit is a legacy alias that
redirects into the details shell | |

#### Alert details (15)

| File | Test | Status |
|---|---|---|
| `details/actions.spec.ts` | AD-06 enable/disable toggle changes the
rule state | |
| `details/actions.spec.ts` | AD-07 Duplicate creates a copy and
navigates to overview | |
| `details/actions.spec.ts` | AD-08 Delete removes the rule and returns
to the list | |
| `details/chrome.spec.ts` | AD-09 copy-link button copies the current
URL to clipboard | conditional |
| `details/chrome.spec.ts` | AD-10 breadcrumb navigates back to the
alert list | |
| `details/chrome.spec.ts` | AD-13 document title updates to show the
rule name | |
| `details/header.spec.ts` | AD-01 v2 header shows editable name input
without Rename menu item | |
| `details/header.spec.ts` | AD-02 v1 header shows static title with
state, severity and labels | |
| `details/not-found.spec.ts` | AD-11 invalid ruleId shows AlertNotFound
page | |
| `details/not-found.spec.ts` | AD-12 missing ruleId on overview shows
AlertNotFound page | |
| `details/rename.spec.ts` | AD-03 v1 rename via modal updates the rule
name | |
| `details/rename.spec.ts` | AD-04 v2 inline rename saves via Overview
footer button | |
| `details/tabs.spec.ts` | AD-05 Overview/History tabs preserve ruleId
and relativeTime | |
| `details/tabs.spec.ts` | AD-05b switching to History tab discards
other history params | |
| `details/threshold-persistence.spec.ts` | TC-02 edit page displays the
saved threshold value | |

#### Alert history (75)

| File | Test | Status |
|---|---|---|
| `history/cross-cutting.spec.ts` | AX-01 full deep-link with all params
is honoured in one load | |
| `history/cross-cutting.spec.ts` | AX-02 page reload preserves all
history params | |
| `history/cross-cutting.spec.ts` | AX-03 browser back/forward restores
correct table state | |
| `history/cross-cutting.spec.ts` | AX-04 no unhandled console errors
across full history session | |
| `history/cross-cutting.spec.ts` | AX-05 no request storm on mount
(exactly one call per endpoint) | |
| `history/cross-cutting.spec.ts` | AX-06 v1 and v2 schema rules both
render history correctly | |
| `history/cross-cutting.spec.ts` | AX-07 no legacy v1 history API calls
during full session | |
| `history/cross-cutting.spec.ts` | AX-08 history API endpoints carry
expected params | |
| `history/empty-and-errors.spec.ts` | AE-01 invalid filter expression
shows syntax error and recovers on fix | |
| `history/empty-and-errors.spec.ts` | AE-02 empty filter_keys response
still mounts editor (no suggestions) | |
| `history/empty-and-errors.spec.ts` | AE-02b bogus ruleId never reaches
history APIs (shows AlertNotFound) | |
| `history/empty-and-errors.spec.ts` | AE-03 rule with no history
renders empty state (not error) | |
| `history/empty-and-errors.spec.ts` | AE-04 time range with no data
renders empty state | |
| `history/empty-and-errors.spec.ts` | AE-05 time-range change resets
pagination to first page | |
| `history/empty-and-errors.spec.ts` | AE-06 absurd time range (90d)
still renders | |
| `history/empty-and-errors.spec.ts` | AE-07 disabled rule history is
still readable | |
| `history/empty-and-errors.spec.ts` | AE-08 deleted rule shows
AlertNotFound on revisit | |
| `history/expression-filter.spec.ts` | AF-06 key suggestions load on
page load | |
| `history/expression-filter.spec.ts` | AF-07 value suggestions fetch
from filter_values endpoint | |
| `history/expression-filter.spec.ts` | AF-08 value suggestions filter
client-side as user types | |
| `history/expression-filter.spec.ts` | AF-09 running equality
expression filters the table | |
| `history/expression-filter.spec.ts` | AF-10 running expression resets
pagination to first page | |
| `history/expression-filter.spec.ts` | AF-11 Run button re-fetches
unchanged expression | |
| `history/expression-filter.spec.ts` | AF-12 in-flight query can be
cancelled | |
| `history/expression-filter.spec.ts` | AF-13 threshold.name and
severity keys filter correctly | |
| `history/expression-filter.spec.ts` | AF-14 unknown key returns 200
with zero rows (not 500) | |
| `history/expression-filter.spec.ts` | AF-15 expression is lost on
Overview→History round-trip (known bug) | |
| `history/expression-filter.spec.ts` | AF-16 expression and state
filter compose in request | |
| `history/expression-filter.spec.ts` | AF-17 clearing expression
restores full unfiltered list | |
| `history/state-filter.spec.ts` | AF-01 All filter sends no state param
in request | |
| `history/state-filter.spec.ts` | AF-02 Fired filter sends state=firing
in request | |
| `history/state-filter.spec.ts` | AF-03 Resolved filter shows empty for
rule with no resolutions | |
| `history/state-filter.spec.ts` | AF-03b Resolved filter shows rows for
rule with resolutions | |
| `history/state-filter.spec.ts` | AF-04 deep-link ?timelineFilter=FIRED
starts on Fired tab | |
| `history/state-filter.spec.ts` | AF-05 changing state filter resets
pagination to first page | |
| `history/statistics.spec.ts` | AS-01 Total Triggered card shows the
firing count | |
| `history/statistics.spec.ts` | AS-02 Avg. Resolution Time card shows
"No Resolutions." when none exist | |
| `history/statistics.spec.ts` | AS-03 empty stats card never renders a
sparkline | |
| `history/statistics.spec.ts` | AS-03b sparkline present with a
multi-point series | skipped |
| `history/statistics.spec.ts` | AS-04 change-vs-past indicator shows
"no previous data" when unavailable | |
| `history/statistics.spec.ts` | AS-09 stats update when time range
changes | |
| `history/statistics.spec.ts` | AS-11 Avg. Resolution Time shows
formatted duration when resolutions exist | |
| `history/statistics.spec.ts` | AS-12 Total Triggered counts only
firing rows (not resolved) | |
| `history/timeline-graph.spec.ts` | AT-03 renders canvas with two
segments (inactive→firing) | |
| `history/timeline-graph.spec.ts` | AT-03b renders canvas with three
segments (inactive→firing→inactive) | |
| `history/timeline-graph.spec.ts` | AT-19 handles nodata state without
console errors | |
| `history/timeline-pagination.spec.ts` | AT-06 next page sends cursor
and shows different rows | |
| `history/timeline-pagination.spec.ts` | AT-07 prev page drops the
cursor from request | |
| `history/timeline-pagination.spec.ts` | AT-08 pagination buttons
disable at first and last page | |
| `history/timeline-pagination.spec.ts` | AT-09 browser back after
paging returns to previous page | |
| `history/timeline-pagination.spec.ts` | AT-10 deep-link ?page=2 loads
second page directly | |
| `history/timeline-pagination.spec.ts` | AT-11 default sort order is
ascending | |
| `history/timeline-pagination.spec.ts` | AT-12 sorting toggles order
and resets to first page | |
| `history/timeline-pagination.spec.ts` | AT-13 single page disables
both pagination buttons | |
| `history/timeline-pagination.spec.ts` | AT-21 all pages together cover
the complete row set | |
| `history/timeline-table.spec.ts` | AT-01 timeline section renders all
chrome elements | |
| `history/timeline-table.spec.ts` | AT-02 Top 5 Contributors tab is
disabled with Coming Soon indicator | |
| `history/timeline-table.spec.ts` | AT-04 table rows display state,
labels and formatted timestamp | |
| `history/timeline-table.spec.ts` | AT-05 footer shows correct row
range | |
| `history/timeline-table.spec.ts` | AT-14 row click does not navigate
away | |
| `history/timeline-table.spec.ts` | AT-15 row actions link navigates to
logs explorer | |
| `history/timeline-table.spec.ts` | AT-15b row actions link navigates
to traces explorer | |
| `history/timeline-table.spec.ts` | AT-16 metrics rule rows show
disabled action (no related links) | |
| `history/timeline-table.spec.ts` | AT-17 CREATED AT column respects
app timezone setting | |
| `history/timeline-table.spec.ts` | AT-18 state cell renders Firing,
Resolved, and No Data correctly | |
| `history/timeline-table.spec.ts` | AT-18b pending/recovering states
render blank (coverage gap) | skipped |
| `history/timeline-table.spec.ts` | AT-18c disabled state renders as
"Muted" (coverage gap) | skipped |
| `history/timeline-table.spec.ts` | AT-20 time-range boundaries
inclusive/exclusive (coverage gap) | skipped |
| `history/top-contributors.spec.ts` | AS-05 card displays max 3 rows
with count ratios | |
| `history/top-contributors.spec.ts` | AS-13 contributor bar width is
the count as a percentage of the total | |
| `history/top-contributors.spec.ts` | AS-06 "View all" button only
appears when more than 3 contributors | |
| `history/top-contributors.spec.ts` | AS-07 View-all drawer shows
paginated list of all contributors | |
| `history/top-contributors.spec.ts` | AS-07b drawer opens from deep
link with ?viewAllTopContributors=true | |
| `history/top-contributors.spec.ts` | AS-08 View-all click adds
?viewAllTopContributors=true to URL | |
| `history/top-contributors.spec.ts` | AS-10 contributor rows show
related-logs link for logs-based rules | |

#### Notification channels (1)

| File | Test | Status |
|---|---|---|
| `channels/edit.spec.ts` | NC-01 an edited recipient persists after
reload | |

#### Skipped tests

| Test | File | Kind | Reason |
|---|---|---|---|
| the v2 Discard button is clickable | `create/edge.spec.ts` | hard
`test.skip(` | Real bug: the button is not clickable. Test written, left
ready to flip. |
| broadcast-to-all saves the rule with the broadcast flag |
`create/v1.spec.ts` | hard `test.skip(` | Real bug: the broadcast flag
is not persisted. |
| sparkline present with a multi-point series |
`history/statistics.spec.ts` | `test.skip(true)` | Flaky by
construction: the sparkline only renders with more than one data point,
and whether the seeded ~2-minute window lands in one stats bucket or two
depends on where it falls relative to the bucket boundary. |
| pending/recovering states render blank |
`history/timeline-table.spec.ts` | `test.skip(true)` | Unreachable:
`pending` and `recovering` are transient states, and no fixture can
reliably catch a rule mid-transition. |
| disabled state renders as "Muted" | `history/timeline-table.spec.ts` |
`test.skip(true)` | Unreachable: a `disabled` history row is
policy-driven, and disabling a rule appends no row (verified). |
| time-range boundaries inclusive/exclusive |
`history/timeline-table.spec.ts` | `test.skip(true)` | Unreachable:
asserting a row exactly at `start` and one at `start-1ms` means
controlling row timestamps, but evaluation times are whatever the ruler
chose. |
| the anomaly card rewrites the rule type, not the alert type |
`create/shell.spec.ts` | conditional | Runs only where the
`ANOMALY_DETECTION` feature flag is active; it is off on this stack. |
| copy-link button copies the current URL to clipboard |
`details/chrome.spec.ts` | conditional | Runs on Chromium only —
Playwright grants `clipboard-read` nowhere else. |


#### Issues closed by this PR
> Reference issues using `Closes #issue-number` to enable automatic
closure on merge.

Closes https://github.com/SigNoz/engineering-pod/issues/4917

---

### ✅ Change Type
_Select all that apply_

- [ ] ✨ Feature
- [ ] 🐛 Bug fix
- [ ] ♻️ Refactor
- [ ] 🛠️ Infra / Tooling
- [x] 🧪 Test-only

---

### ⚠️ Risk & Impact Assessment
> What could break? How do we recover?

- Blast radius: Alerts
- Potential regressions: None, only test ids
- Rollback plan: Find and fix the issue specifically

---

### 📝 Changelog
> Fill only if this affects users, APIs, UI, or documented behavior  
> Use **N/A** for internal or non-user-facing changes

| Field | Value |
|------|-------|
| Deployment Type | Cloud / OSS / Enterprise |
| Change Type | Maintenance |
| Description | We added more E2E tests for Alerts page. |

---

### 📋 Checklist
- [x] Tests added or explicitly not required
- [ ] Manually tested
- [ ] Breaking changes documented
- [ ] Backward compatibility considered
 2026-08-26 06:37:32 +00:00
Pandey
5917f9fe31 perf(tests): cache go and pnpm stores across integration image builds (#12366) 
* perf(tests): cache go and pnpm stores across integration image builds

Add BuildKit cache mounts for GOCACHE/GOMODCACHE and the pnpm store to the
integration Dockerfiles, and build the image via the docker CLI (docker-py,
used by testcontainers' DockerImage, does not support BuildKit). Embed the
go build command directly so Makefile changes do not invalidate the build
layer, and pin HOME/GOCACHE/GOMODCACHE/PNPM_HOME explicitly so cache-mount
targets match tool defaults by contract. The with-web node stage fetches
dependencies from the lockfile before the source copy, so frontend edits
only re-run the offline install and build.

* feat(tests): add --clean flag to prune buildkit cache mounts

The go and pnpm caches introduced for the integration image build survive
--teardown since they belong to the docker builder, not to any container.
--clean runs docker builder prune with a type=exec.cachemount filter at
session start, forcing the next image build to start cold. Documented in
the integration testing guide.

* feat(tests): add --rebuild flag to refresh the signoz container under --reuse

--reuse keeps the running signoz container, so backend source changes are
never picked up without tearing down the whole stack. --rebuild deletes the
cached signoz container and recreates it from the current sources (an
incremental image build), while databases, mocks and migrations stay reused.
Requires --reuse; combining with --teardown or --clean is a usage error.

* chore(tests): prune comments to non-obvious constraints

* docs(tests): make py-test-setup rebuild signoz and audit the integration guide

py-test-setup now passes --rebuild so re-running it after backend changes
transparently swaps in a signoz container built from the current sources.
The integration guide documents the iteration loop and fixes stale content:
option defaults (clickhouse 25.12.5, schema migrator v0.144.6), the
nonexistent --zookeeper-version option, Zookeeper vs ClickHouse Keeper, the
e2e doc path, and the lint toolchain (ruff).

* docs(tests): wire --rebuild into the e2e setup flow

The e2e bootstrap shares the signoz fixture, so --rebuild already applies;
with --with-web it also picks up frontend changes since the image bakes the
built frontend in. The setup command now passes --rebuild, and the guide
documents the iteration loop, the --rebuild/--clean flags, ClickHouse Keeper
instead of Zookeeper, and the corrected integration doc path.

* docs(tests): qualify --rebuild workflow for suites with custom signoz variants

make py-test-setup only rebuilds the default signoz instance; suites that
create their own via create_signoz(cache_key=...) keep a separately cached
container. Passing --rebuild on the suite run itself rebuilds every variant
that run instantiates.

* docs(tests): describe --clean behaviour instead of its exact command

Keeps the docs from drifting if the prune invocation behind --clean changes.
 2026-08-01 13:52:19 +00:00
Ashwin Bhatkal
ae2127afe8 test: dashboards list spec with new e2e framework (#11190)
Some checks failed
build-staging / prepare (push) Has been cancelled
Details
build-staging / js-build (push) Has been cancelled
Details
build-staging / go-build (push) Has been cancelled
Details
build-staging / staging (push) Has been cancelled
Details
Release Drafter / update_release_draft (push) Has been cancelled
Details 
* test: dashboards list spec with new e2e framework

* chore: update docker ignore

* test: dashboards list spec with new e2e framework

* test: fix skipped ones

* test: fix scroll

* test: fix flaky clicks

* test: fix formatting

* chore: doc update + ignore file changes

* chore: update fixtures vs helpers

* chore: resolve comments

* chore: resolve comments
 2026-05-07 14:55:38 +00:00
Pandey
93f5df9185 tests: unify integration + e2e under shared pytest project (#11019)
Some checks failed
build-staging / prepare (push) Has been cancelled
Details
build-staging / js-build (push) Has been cancelled
Details
build-staging / go-build (push) Has been cancelled
Details
build-staging / staging (push) Has been cancelled
Details
Release Drafter / update_release_draft (push) Has been cancelled
Details 
* refactor(tests): hoist pytest project to tests/ root for shared fixtures

Lift pyproject.toml, uv.lock, conftest.py, and fixtures/ up from
tests/integration/ so the pytest project becomes shared infrastructure
rather than integration's private property. A sibling tests/e2e/ can
reuse the same fixture graph (containers, auth, seeding) without
duplicating plugins.

Also:
- Merge tests/integration/src/querier/util.py into tests/fixtures/querier.py
  (response assertions and corrupt-metadata generators belong with the
  other querier helpers).
- Use --import-mode=importlib + pythonpath=["."] in pyproject so
  same-basename tests across src/*/ do not collide at the now-wider
  rootdir.
- Broaden python_files to "*/src/**/**.py" so future test trees under
  tests/e2e/src/ get discovered.
- Update Makefile py-* targets and integrationci.yaml to cd into tests/
  and reference integration/src/... paths.

* feat(tests/e2e): import Playwright suite from signoz-e2e

Relocate the standalone signoz-e2e repository into tests/e2e/ as a
sibling of tests/integration/. The suite still points at remote
staging by default; subsequent commits wire it to the shared pytest
fixture graph so the backend can be provisioned locally.

Excluded from the import: .git, .github (CI migration deferred),
.auth, node_modules, test-results, playwright-report.

* feat(tests/e2e): pytest-driven backend bring-up, seeding, and playwright runner

Wire the Playwright suite into the shared pytest fixture graph so the
backend + its seeded state are provisioned locally instead of pointing
at remote staging.

Python side (owns lifecycle):
- tests/fixtures/dashboards.py — generic create/list/upsert_dashboard
  helpers (shared infra; testdata stays per-tree).
- tests/e2e/conftest.py — e2e-scoped pytest fixtures: seed_dashboards
  (idempotent upsert from tests/e2e/testdata/dashboards/*.json),
  seed_alert_rules (from tests/e2e/testdata/alerts/*.json, via existing
  create_alert_rule), seed_e2e_telemetry (fresh traces/logs across a
  few synthetic services so /home and Services pages have data).
- tests/e2e/src/bootstrap/setup.py — test_setup depends on the fixture
  graph and persists backend coordinates to tests/e2e/.signoz-backend.json;
  test_teardown is the --teardown target.
- tests/e2e/src/bootstrap/run.py — test_e2e: one-command entrypoint that
  brings up the backend + seeds, then subprocesses yarn test and asserts
  Playwright exits 0.
- tests/conftest.py — register fixtures.dashboards plugin.

Playwright side (just reads):
- tests/e2e/global.setup.ts — loads .signoz-backend.json and injects
  SIGNOZ_E2E_BASE_URL/USERNAME/PASSWORD. No-op when env is already
  populated (staging mode, or pytest-driven runs where env is pre-set).
- playwright.config.ts registers globalSetup.
- package.json gains test:staging; existing scripts unchanged.

Testdata layout: tests/e2e/testdata/{dashboards,alerts,channels}/*.json
— per-tree (integration has its own tests/integration/testdata/).

* docs(tests): describe pytest-master workflow and shared fixture layout

- tests/README.md (new): top-level map of the shared pytest project,
  fixture-ownership rule (shared vs per-tree), and common commands.
- tests/e2e/README.md: lead with the one-command pytest run and the
  warm-backend dev loop; keep the staging fallback as option 2.
- tests/e2e/CLAUDE.md: updated commands so agent contexts reflect the
  pytest-driven lifecycle.
- tests/e2e/.env.example: drop unused SIGNOZ_E2E_ENV_TYPE; note the file
  is only needed for staging mode.

* fix(tests/fixtures/signoz.py): anchor Docker build context to repo root

Previously used path="../../" which resolved to the repo root only when
pytest's cwd was tests/integration/. After hoisting the pytest project
to tests/, that same relative path pointed one level above the repo
root and the build failed with:

  Cannot locate specified Dockerfile: cmd/enterprise/Dockerfile.with-web.integration

Anchor the build context to an absolute path computed from __file__ so
the fixture works regardless of pytest cwd.

* feat(tests/e2e): alerts-downtime regression suite (platform-pod/issues/2095)

Import the 34-step regression suite originally developed on
platform-pod/issues/2095-frontend. Targets the alerts and planned-downtime
frontend flows after their migration to generated OpenAPI clients and
generated react-query hooks.

- specs/alerts-downtime/: SUITE.md (the stable spec), README.md (scope +
  open observations from the original runs), results-schema.md (legacy
  per-run artifact shape, retained for context).
- tests/alerts-downtime/alerts-downtime.spec.ts: 881-line Playwright spec
  covering 6 flows — alert CRUD/toggle, alert detail 404, planned
  downtime CRUD, notification channel routing, anomaly alerts.

Integration with the shared suite:
- Uses baseURL + storageState from tests/e2e/playwright.config.ts (no
  separate config). page.goto calls use relative paths; SIGNOZ_E2E_*
  env vars from the pytest bootstrap drive auth.
- test.describe.configure({ mode: 'serial' }) at the top of the describe:
  the flows mutate shared tenant state, so parallel runs cause cross-
  flow interference (documented in the original 2095 config).
- Per-run artifacts (network captures + screenshots) land in
  tests/e2e/tests/alerts-downtime/run-spec-<ts>/ by default — gitignored.

Historical per-run artifacts (~7.5MB of screenshots across run-1 through
run-7) are not imported; they lived at e2e/2095/run-*/ on the original
branch and remain there if needed.

* refactor(fixtures/traces): extract insert + truncate helpers

Pull the ClickHouse insert path out of the insert_traces pytest fixture
into a plain module-level function insert_traces_to_clickhouse(conn,
traces), and move the per-table TRUNCATE loop into truncate_traces_tables
(conn, cluster). The fixture becomes a thin wrapper over both — zero
behavioural change.

Lets the HTTP seeder container (tests/fixtures/seeder/) reuse the exact
same insert + truncate code the pytest fixture uses, so the two stay in
sync as the trace schema evolves.

* feat(fixtures/seeder): HTTP seeder container for fine-grained telemetry seeding

Adds a sibling container alongside signoz/clickhouse/postgres that exposes
HTTP endpoints for direct-ClickHouse telemetry seeding, so Playwright
tests can shape per-test data without going through OTel or the SigNoz
ingestion path.

tests/fixtures/seeder/:
- Dockerfile: python:3.13-slim + the shared fixtures/ tree so the
  container can import fixtures.traces and reuse the exact insert path
  used by pytest.
- server.py: FastAPI app with GET /healthz, POST /telemetry/traces
  (accepts a JSON list matching Traces.from_dict input; auto-tags each
  inserted row with resource seeder=true), DELETE /telemetry/traces
  (truncates all traces tables).
- requirements.txt: fastapi, uvicorn, clickhouse-connect, numpy plus
  sqlalchemy/pytest/testcontainers because fixtures/{__init__,types,
  traces}.py import them at module load.

tests/fixtures/seeder/__init__.py: pytest fixture (`seeder`, package-
scoped) that builds the image via docker-py (testcontainers DockerImage
had multi-segment dockerfile issues), starts the container on the
shared network wired to ClickHouse via env vars, and waits for
/healthz. Cache key + restore follow the dev.wrap pattern other
fixtures use for --reuse.

tests/.dockerignore: exclude .venv, caches, e2e node_modules, and test
outputs so the build context is small and deterministic.

tests/conftest.py: register fixtures.seeder as a pytest plugin.

Currently traces-only — logs + metrics follow the same pattern.

* feat(tests/e2e): surface seeder_url to Playwright via globalSetup

- bootstrap/setup.py: test_setup now depends on the seeder fixture and
  writes seeder_url into .signoz-backend.json alongside base_url.
- bootstrap/run.py: test_e2e exports SIGNOZ_E2E_SEEDER_URL to the
  subprocessed yarn test so Playwright specs can reach the seeder
  directly in the one-command path.
- global.setup.ts: if .signoz-backend.json carries seeder_url, populate
  process.env.SIGNOZ_E2E_SEEDER_URL. Remains optional — staging mode
  leaves it unset.

Playwright specs that want per-test telemetry can:
  await fetch(process.env.SIGNOZ_E2E_SEEDER_URL + '/telemetry/traces', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify([...])
  });
and await a truncate via DELETE on teardown.

* fix(alerts-downtime): capture load-time GETs before navigation

Flow 1 registered cap.mark() AFTER page.goto() and then called
page.waitForResponse(/api/v2/rules) — but against a fast local backend
the GET /api/v2/rules response arrived during page.goto, before the
waiter could register, and the test timed out at 30s.

installCapture's page.on('response') listener runs from before the
navigation, so moving mark() above page.goto() and relying on
dumpSince's 500ms drain is enough. No lost precision.

One site only; the same pattern exists in later flows (via per-action
waitForResponse) and may surface similar races — those are left for a
follow-up once the backend-side 2095 migration lands on main (current
frontend still calls PATCH /api/v1/rules/:id which the spec's assertion
doesn't match anyway).

* refactor(fixtures/logs,metrics): extract insert + truncate helpers

Mirror the traces refactor: pull the ClickHouse insert path out of the
insert_logs / insert_metrics pytest fixtures into plain module-level
functions (insert_logs_to_clickhouse, insert_metrics_to_clickhouse) and
move the per-table TRUNCATE loops into truncate_logs_tables /
truncate_metrics_tables. The fixtures become thin wrappers — zero
behavioural change.

Sets up the seeder container to expose POST/DELETE endpoints for logs
and metrics using the exact same code paths as the pytest fixtures.

* feat(fixtures/seeder): add logs and metrics endpoints

Extend the seeder with POST/DELETE endpoints for logs and metrics,
following the same shape as the existing traces endpoints:

- POST /telemetry/logs accepts a JSON list matching Logs.from_dict;
  tags each row's resources with seeder=true.
- POST /telemetry/metrics accepts a JSON list matching Metrics.from_dict;
  tags resource_attrs with seeder=true (Metrics.from_dict unpacks
  resource_attrs rather than a resources dict).
- DELETE /telemetry/logs, DELETE /telemetry/metrics truncate via the
  shared truncate_*_tables helpers.

Requirements gain svix-ksuid because fixtures/logs.py imports KsuidMs
for log id generation.

Verified end-to-end against the warm backend: POST inserted=1 on each
signal, DELETE truncated=true on each.

* refactor(fixtures/seeder): align status codes with HTTP semantics

- POST /telemetry/{traces,logs,metrics}: return 201 Created (kept the
  {inserted: N} body so callers can verify the count landed).
- DELETE /telemetry/{traces,logs,metrics}: return 204 No Content with
  an empty body.

* refactor(tests/seeder): extract from fixtures/ into top-level package

Move the HTTP seeder (Dockerfile, requirements.txt, server.py) out of
tests/fixtures/seeder/ and into its own tests/seeder/ top-level package.
The pytest fixture that builds and runs the image moves to
tests/fixtures/seeder.py so it sits next to the other container fixtures.

Rationale: the seeder is a standalone containerized Python service, not a
pytest fixture. It ships a Dockerfile, its own requirements.txt, and a
server.py entrypoint — none of which belong under a package whose purpose
is shared pytest code.

Image-side changes:
- Dockerfile now copies seeder/ alongside fixtures/ and launches
  seeder.server:app instead of fixtures.seeder.server:app.
- Build context stays tests/ (unchanged), so fixtures.* imports inside
  server.py continue to resolve.

Fixture-side changes:
- _TESTS_ROOT computation drops one parent (parents[1] now that the file
  is at tests/fixtures/seeder.py, not tests/fixtures/seeder/__init__.py).
- The dockerfile= path passed to docker-py becomes seeder/Dockerfile.

No behavior change; every consumer still imports the seeder fixture as
before and gets the same container.

* refactor(fixtures/keycloak): rename from idp.py to name the concrete tech

The container provider at fixtures/idp.py brought up a Keycloak image. Name
it for what it is so we can use fixtures/idp.py later for API-side IdP
helpers (OIDC/SAML admin flows) without an idp-vs-idputils naming collision.

- fixtures/idp.py → fixtures/keycloak.py (git rename).
- fixtures.idputils updates its one internal import to fixtures.keycloak.
- conftest.py pytest_plugins entry points at the new module.

No caller outside fixtures/ imports fixtures.idp directly, so no shim is
needed. The "idp" fixture name (how tests reference it) is unchanged.

* refactor(fixtures/gateway): drop -utils suffix

The module only held helper functions (no fixtures). Rename to match the
domain and leave a shim at the old path so integration/ import sites keep
working until they are swept in a follow-up.

* fix(fixtures/gatewayutils): silence wildcard-import in deprecation shim

The shim intentionally re-exports via `from fixtures.gateway import *`;
pylint flags the wildcard and every unused-wildcard symbol. Suppress both
in the shim only — the live module has no wildcard.

* refactor(fixtures/auth): merge authutils helpers into auth

Pull the pure-helper functions from authutils.py (create_active_user,
find_user_by_email, find_user_with_roles_by_email, assert_user_has_role,
change_user_role) into auth.py next to the fixtures they complement.
Fixtures remain on top; helpers go below. Drop the module docstring.

Replace authutils.py with a deprecation shim that re-exports from
fixtures.auth so integration/ import sites (9 files) keep working until
they are swept in a follow-up. Suppress the wildcard-import warnings in
the shim only.

* refactor(fixtures/alerts): merge alertutils helpers into alerts

Pull the pure-helper functions from alertutils.py
(collect_webhook_firing_alerts, _verify_alerts_labels,
verify_webhook_alert_expectation, update_rule_channel_name) into alerts.py
next to the fixtures they complement. Fixtures stay on top; helpers go
below.

Replace alertutils.py with a deprecation shim that re-exports from
fixtures.alerts so integration/ import sites keep working until they are
swept in a follow-up.

* refactor(fixtures/cloudintegrations): merge cloudintegrationsutils helpers

Pull the pure-helper functions from cloudintegrationsutils.py
(deprecated_simulate_agent_checkin, setup_create_account_mocks,
simulate_agent_checkin) into cloudintegrations.py next to the fixtures
they complement. Fixtures stay on top; helpers go below.

Replace cloudintegrationsutils.py with a deprecation shim that re-exports
from fixtures.cloudintegrations so integration/ import sites keep working
until they are swept in a follow-up.

* refactor(fixtures/idp): rename idputils to idp now that keycloak owns the container

With the Keycloak container provider at fixtures.keycloak, the fixtures.idp
name is free for what idputils always was — API/browser helpers for OIDC
and SAML admin flows against the IdP container.

- fixtures.idputils → fixtures.idp (git rename).
- conftest.py pytest_plugins swaps fixtures.idputils for fixtures.idp so
  the create_saml_client / create_oidc_client fixtures register under the
  canonical path.

Replace fixtures.idputils with a deprecation shim re-exporting from
fixtures.idp so integration/ import sites (callbackauthn) keep working
until they are swept in a follow-up.

* refactor(fixtures/reuse): rename from dev to describe what the module is

The module wraps pytest-cache resource reuse/teardown for container
fixtures; "dev" conveyed nothing about its role. Rename to fixtures.reuse
and update the 12 internal callers that imported `from fixtures import
dev, types` to use `reuse` instead.

Replace fixtures.dev with a deprecation shim so any external caller keeps
working until the follow-up sweep.

* refactor(fixtures/time,fs): split utils by responsibility

fixtures.utils only held two time parsers (parse_timestamp, parse_duration)
and one path helper (get_testdata_file_path) — a "utils" grab bag.

- Time parsers move to fixtures.time (utils.py → time.py via git rename).
- get_testdata_file_path moves into fixtures.fs where other filesystem
  helpers live.
- Internal callers (alerts, logs, metrics, traces) update to the new paths.

Replace fixtures.utils with a deprecation shim that re-exports all three
functions so integration/ import sites keep working until the follow-up
sweep.

* refactor(fixtures/browser): rename from driver to match peer primitives

fixtures.driver was the Selenium WebDriver fixture — rename the module to
fixtures.browser so it sits next to fixtures.http as a named primitive.
The fixture name inside (driver) stays — that's the Selenium-canonical
term and tests reference it directly.

conftest.py pytest_plugins entry points at the new module. A deprecation
shim at fixtures.driver keeps any external caller working until the
follow-up sweep.

* refactor(tests/seeder): install deps via uv from pyproject, drop requirements.txt

The seeder's requirements.txt duplicated 7 of 10 deps from pyproject.toml
with overlapping version pins — a standing drift risk. The comment on top
of the file admitted the real problem: the seeder image already ships
pytest + testcontainers + sqlalchemy because importing fixtures.traces
walks fixtures/__init__.py and fixtures/types.py. "Don't ship test infra"
was already violated.

- Add fastapi, uvicorn[standard], and py to pyproject.toml dependencies
  (the three seeder-only deps that were not yet in pyproject; `py` was a
  latent gap since fixtures/types.py uses py.path.local but pytest only
  pulls it in transitively).
- Switch the Dockerfile to `uv sync --frozen --no-install-project --no-dev`
  so the container env matches local dev exactly (uv.lock is the single
  source of truth for versions).
- Move tests/seeder/Dockerfile → tests/Dockerfile.seeder so it lives
  alongside the pyproject at the root of the build context.
- Delete tests/seeder/requirements.txt.

The seeder image grows by ~40-50MB (selenium, psycopg2, wiremock now come
along from main deps); accepted as a cost of single source of truth since
the seeder is dev-only infra, not a shipped artifact.

* refactor(tests/integration): flatten src/ into bootstrap/ + tests/

Drop the redundant src/ layer in the integration tree. 'src' carries no
information — the directory IS integration test source. After flatten:

  tests/integration/
    bootstrap/setup.py        was src/bootstrap/setup.py
    tests/<suite>/*.py        was src/<suite>/*.py (16 suites)
    testdata/

Updates:
- Makefile: py-test-setup/py-test-teardown/py-test target paths.
- tests/README.md: layout diagram + command examples.
- tests/pyproject.toml: python_files glob now matches basenames
  explicitly — "[0-9][0-9]_*.py" for NN-prefixed suite files plus
  "setup.py" and "run.py" for bootstrap entrypoints. The old "*/src/.."
  glob stopped matching anything here and would have caused pytest to
  try collecting seeder/server.py as a test.

* refactor(tests/e2e): flatten src/ into bootstrap/

Drop the e2e/src/ wrapper — the only Python content under it was
bootstrap/, which is now a direct child of e2e/. Keeps integration and
e2e symmetric (both have bootstrap/, tests/, testdata/ as peers).

Also delete bootstrap/__init__.py on both integration and e2e sides.
With --import-mode=importlib, pytest walks up from each .py file to find
the highest __init__.py-containing dir and uses that as the package root.
Without integration/__init__.py or e2e/__init__.py above bootstrap/, both
setup.py files resolved to the same dotted name `bootstrap.setup`, causing
a sys.modules collision that silently dropped test_telemetry_databases_exist
from integration's bootstrap. With no __init__.py anywhere, pytest treats
each setup.py as a standalone module via spec_from_file_location and both
are collected cleanly.

Updates tests/README.md, tests/e2e/README.md, and tests/e2e/CLAUDE.md path
references from e2e/src/bootstrap/ to e2e/bootstrap/.

* refactor(tests/e2e): drop specs/ + strip // spec: back-pointers

specs/ held markdown test plans that mirrored tests/ 1:1 as pre-code
scratch. Once a test exists, the plan is stale the moment the test
diverges — they're AI-planner output, not source of truth. Keep the
workflow alive by .gitignore-ing specs/ (the planner agent can still
write locally) but stop shipping stale plans in the repo.

Strip the `// spec: specs/...` and `// seed: tests/seed.spec.ts` header
comments from 5 .spec.ts files. The spec pointer is dead; the seed
pointer was convention-only — Playwright collects regardless.

* docs(contributing/tests): move e2e/integration guides out of test dirs

Pull the e2e contributor guide out of tests/e2e/CLAUDE.md (which read
like a full agent-workflow reference doc) and into
docs/contributing/tests/e2e.md alongside the existing development / go
guides.

- Delete tests/e2e/CLAUDE.md; its content (layout, commands, role tags,
  locator priority, Playwright agent workflow) lives in the new e2e.md
  with references to the now-.gitignore'd specs/ dir removed.
- Add docs/contributing/tests/integration.md — short guide covering
  layout, runner commands, filename conventions, and the flow for
  adding a new suite (there was no contributor doc for this before).
- Trim tests/e2e/README.md to quick-start + commands; link out to the
  full guide. Readers who just want to run tests get the 5 commands
  they need; anything deeper is one hop away.

* chore(tests/e2e): drop examples/example-test-plan.md

Init-agents boilerplate. Fresh planner agents don't need a checked-in
template; they can write to the .gitignore'd specs/ scratch dir.

tests/integration/.qodo/ was also removed (untracked, empty; .qodo is
already in the root .gitignore).

* refactor(tests/seeder): use fixtures.logger.setup_logger

Drop the one-off logging.basicConfig + logging.getLogger("seeder") in
favor of the shared setup_logger helper that every fixtures/*.py already
uses. Keeps log format consistent across pytest runs and the seeder
container.

fixtures.logger ships into the image via the existing COPY fixtures step
in Dockerfile.seeder — no build change needed.

* fix(tests/e2e): correct e2e_dir path after src/ flatten

After phase 2 (flatten tests/e2e/src/ into tests/e2e/), the run.py file
sits one level closer to the e2e root. parents[2] now resolves to tests/
instead of tests/e2e/, so yarn test would subprocess from the wrong cwd.

parents[1] is the correct index now.

* fix(tests/e2e): correct endpoint-file path in setup.py after src/ flatten

Same class of stale-path bug as the run.py fix: after the e2e/src/
flatten, setup.py sits one level closer to the e2e root. parents[2] now
lands at tests/ instead of tests/e2e/, so .signoz-backend.json would be
written to tests/.signoz-backend.json and the Playwright global.setup.ts
(which expects tests/e2e/.signoz-backend.json) wouldn't find it.

parents[1] is correct.

* refactor(tests/e2e): drop pre-seed fixtures; each spec owns its data

The seeder (tests/seeder/) was built so specs can POST telemetry
per-test. Global pre-seeding via tests/e2e/conftest.py (seed_dashboards,
seed_alert_rules, seed_e2e_telemetry) is the exact anti-pattern that
setup obsoletes — shared state across specs, order-dependent runs, no
reset between tests.

- Delete tests/e2e/conftest.py (3 fixtures, all pre-seed).
- Delete tests/e2e/testdata/dashboards/apm-metrics.json — its only
  consumer was seed_dashboards. tests/e2e/testdata/ now empty and gone.
- Drop seed_dashboards, seed_alert_rules, seed_e2e_telemetry params
  from bootstrap/setup.py::test_setup and bootstrap/run.py::test_e2e.
  test_teardown never depended on them.
- Refresh the module docstrings on both bootstrap tests to reflect the
  new model (backend + seeder up; specs seed themselves).
- Update tests/README.md and docs/contributing/tests/e2e.md: remove the
  testdata/ + conftest.py references, document the per-spec seeding
  rule (telemetry via seeder endpoints, dashboards/alerts via SigNoz
  REST API from the spec).

Known breakage: tests/e2e/tests/dashboards/dashboards-list.spec.ts
expects at least one dashboard to exist. With seed_dashboards gone, it
will fail until that spec is updated to create its own dashboard via
the SigNoz API in test.beforeAll. Followup.

* refactor(tests/e2e): relocate auth helper into fixtures/; expose authedPage

Rename tests/e2e/utils/login.util.ts → tests/e2e/fixtures/auth.ts and
drop the (now-empty) utils/ dir. "Fixtures" is the unit of per-test
shared setup on both the Python and TS sides of this project — naming
them consistently across trees makes the parallel obvious.

fixtures/auth.ts now exports three things:

- `test` — Playwright test extended with an authedPage fixture. New
  specs can request `authedPage` as a param and skip the
  `beforeEach(() => ensureLoggedIn(page))` boilerplate entirely.
- `expect` — re-exported from @playwright/test so callers have one
  import.
- `ensureLoggedIn(page)` — the underlying helper, still exported for
  specs that want per-call control.

Update the 4 specs that imported from utils/login.util to point at the
new path; no behavior change in those specs (they keep calling
ensureLoggedIn in beforeEach). Refactoring them to use authedPage can
happen spec-by-spec later.

Also update the path example in .cursorrules so AI-generated snippets
reach for the new import path.

* refactor(tests/e2e): emit .env.local instead of .signoz-backend.json

The old flow (pytest writes JSON → global.setup.ts loads it → exports
env vars) was doing what dotenv already does. Collapse to the native
pattern:

- bootstrap/setup.py writes tests/e2e/.env.local with the four coords
  (BASE_URL, USERNAME, PASSWORD, SEEDER_URL). File header marks it as
  generated.
- playwright.config.ts loads .env first, then .env.local with
  override=true. User-provided defaults stay in .env; generated values
  win when present.
- Delete tests/e2e/global.setup.ts (36 lines gone) and its globalSetup
  reference in playwright.config.ts.

Subprocess-injected env (run.py shelling out to yarn test) still wins
because dotenv doesn't overwrite already-set process.env keys.

Rename the test-only override env var SIGNOZ_E2E_ENDPOINT_FILE →
SIGNOZ_E2E_ENV_FILE for accuracy. Update .env.example, .gitignore (drop
.signoz-backend.json, keep .env.local with its explanatory comment),
tests/README.md, docs/contributing/tests/e2e.md.

* refactor(tests/e2e/alerts-downtime): drop custom network + screenshot capture

The spec wrapped every /api/ response in a bespoke installCapture(), wrote
hand-named JSON files per call (01_step1.1_GET_rules.json, ...), and took
step-by-step screenshots — all going into run-spec-<ts>/ next to the spec
(gitignored).

Playwright already records equivalent data via `trace` (network bodies,
screenshots per step, DOM snapshots, console — viewable via
`playwright show-trace`). The capture infra was duplicating that for the
one-shot 2095 regression audit; no downstream consumer reads the JSON or
PNG artifacts now.

- Remove installCapture, shot, RUN_DIR/NET_DIR/SHOT_DIR, fs/path imports.
- Strip cap.mark()/cap.dumpSince()/shot() calls throughout the 7 flows.
- Collapse the block-scopes that only existed to bound mark variables.
- Drop the "Artifacts" paragraph from the file's top-of-file comment.
- Remove the `tests/alerts-downtime/run-spec-*/` entry from .gitignore.

Spec drops from 885 lines to 736 (≈17% smaller). All 7 flows + their
assertions are unchanged. For debug access, rely on
`trace: 'on-first-retry'` (already set in playwright.config.ts) + `yarn
show-trace`.

* refactor(tests/e2e): move alerts-downtime.spec.ts into alerts/

The spec lives mostly in the alerts domain (6 of 7 flows), with the
planned-downtime CRUD (Flow 4) and cascade-delete (Flow 5) as
cross-feature collateral. The standalone alerts-downtime/ dir was
compound-named, breaking the one-feature-per-dir pattern every other
dir under tests/ follows, and duplicating the spec's own filename.

Move to tests/alerts/alerts-downtime.spec.ts. Empty alerts-downtime/
dir removed.

* refactor(tests/e2e): consolidate Playwright output under artifacts/

All Playwright outputs now land under a single tests/e2e/artifacts/ dir
so CI can archive it in one command (tar / zip / upload-artifact). Each
piece was writing to its own sibling of tests/e2e/ before.

playwright.config.ts:
- outputDir: 'artifacts/test-results' — per-test traces, screenshots,
  videos (was default test-results/).
- HTML reporter → 'artifacts/html-report' (was default
  playwright-report/); open: 'never' so CI doesn't spawn a browser on
  report generation.
- JSON reporter → 'artifacts/results.json' (was
  'test-results/results.json').

package.json: `yarn report` now points playwright show-report at the new
HTML folder.

Ignore updates — replace the two old paths with /artifacts/ in
tests/e2e/.gitignore, tests/e2e/.prettierignore, and tests/.dockerignore
(seeder image build context).

.cursorrules: update the `cat test-results/results.json` example to the
new path so AI-generated snippets reach for the right file.

Delete the empty test-results/ and playwright-report/ dirs that prior
runs left behind.

* refactor(tests/e2e): one artifacts/ subdir per reporter

Within artifacts/, give each reporter its own named subdir so the layout
tells you what wrote what:

  artifacts/
    html/              # HTML reporter (was artifacts/html-report)
    json/results.json  # JSON reporter (was artifacts/results.json)
    test-results/      # outputDir — per-test traces/screenshots/videos

`yarn report` and the .cursorrules cat example point at the new paths.

* refactor(tests/e2e): drop SIGNOZ_USER_ROLE env filter and @admin/@editor/@viewer tags

The filter claimed to be role-based but only grep'd by tag — the actual
browser session is always admin (bootstrap creates one admin, auth.setup.ts
saves one storageState, every project uses it). Tagging tests `@viewer`
didn't mean they ran as a viewer; it just meant they'd be in the subset
selected when SIGNOZ_USER_ROLE=Viewer. Superset semantics (admin sees
everything) meant the filter was at best a narrower test selection and
at worst a misleading assertion of role coverage.

Gone:
- getRoleGrepPattern() + grep: line in playwright.config.ts.
- The dedicated setup project's grep override (no filter to override).
- SIGNOZ_USER_ROLE entries in .env.example, README, docs/contributing.
- The "Role-Based Testing" section + all role-tagging guidance and
  example snippets in .cursorrules.
- All `{ tag: '@viewer' | '@editor' | '@admin' }` annotations on the 90
  affected test sites across 5 spec files (single-line and multi-line
  forms). ~90 annotations gone.

For ad-hoc selection, `yarn test --grep <pattern>` still works on
Playwright's normal grep (test titles/paths).

Real role-based coverage (separate users + storageStates per role) is a
different problem — not pretending this was it.

* chore(tests/e2e): drop .cursorrules

* refactor(tests/e2e): move auth from project-level storageState to per-suite fixture

Replaced auth.setup.ts + globally-mounted storageState with a test-scoped
authedPage fixture in tests/e2e/fixtures/auth.ts. Each suite controls its
own identity via `test.use({ user: ... })`; specs that need to run
unauthenticated just request the stock `page` fixture instead.

fixtures/auth.ts:
- Declares `user` as a test option, defaulting to ADMIN (creds from
  .env.local / .env).
- authedPage resolves to a Page whose context has storageState mounted
  for that user. First request per (user, worker) triggers one login
  and writes a per-user storageState file under .auth/; subsequent
  requests reuse it via a Promise-valued cache.
- Exposes `User` type and `ADMIN` constant so future suites can declare
  additional users (EDITOR, VIEWER) as credentials become available.

playwright.config.ts:
- Drop authFile constant, `setup` project, storageState + dependencies
  on each browser project.

tests/auth.setup.ts:
- Deleted. Login logic now lives inside fixtures/auth.ts's login() helper,
  called on demand by the fixture rather than upfront for the whole run.

Spec migration (6 files):
- Import `test, expect` from ../fixtures/auth (or ../../fixtures/auth)
  instead of @playwright/test.
- Drop `ensureLoggedIn` imports and `await ensureLoggedIn(page)` calls.
- Swap `{ page }` → `{ authedPage: page }` in test and beforeEach
  destructures (local var stays `page` via aliasing so test bodies need
  no further changes).

Cost: N logins per run, where N = unique users × workers (= 1 × 2–4
today, vs the old 1 globally). Tradeoff for explicit per-suite control.

Specs that need unauth later just use `async ({ page }) => ...` — the
fixture isn't invoked, so no login fires.

291 tests still list (previously 292: the old auth.setup.ts counted as
one fake "test"; it's gone now).

* refactor(tests/e2e): cache auth storageState in memory, drop .auth/ dir

The fixture was writing each user's storageState to .auth/<user>.json and
then handing Playwright the file path. But Playwright's
browser.newContext({ storageState }) accepts the object form too —
ctx.storageState() without a path arg returns the cookies+origins
inline.

Keeping the cache in memory means no filesystem roundtrip per login, no
.auth/ dir to maintain, no stale JSON persisting across runs, and no
gitignore entry for it. Each worker's Map holds one Promise<StorageState>
per unique user, resolved on first login and reused thereafter.

Drop the .auth/ entry from tests/e2e/.gitignore; delete the (now unused)
on-disk .auth/ dir.

* chore(tests/e2e): drop seed.spec.ts

* chore(tests/e2e): drop unused README.md and .mcp.json

* refactor(tests/e2e): move existing specs to legacy/ pending fresh rewrite

Park the 5 current spec files under tests/e2e/legacy/ while fresh specs
get written in tests/e2e/tests/ against the new conventions (TC-NN
titles, authedPage fixture, minimal direct-fetch). Playwright's testDir
stays pointed at ./tests — `yarn test` now finds 0 tests until the
first fresh spec lands. legacy/ is preserved for reference but not
collected by default.

Add a .gitkeep under tests/ so the empty dir survives in git between
the move and the first new spec.

Running legacy on demand:
  npx playwright test --config tests/e2e/playwright.config.ts \
    --project chromium legacy/<spec>.ts
(or temporarily point testDir at ./legacy in the config). No yarn
script wired — legacy is expected to rot as fresh specs replace it.

* refactor(tests): drop -utils deprecation shims; import from canonical modules

The shims we introduced during the phase-3 merges (authutils, alertutils,
cloudintegrationsutils, idputils, gatewayutils) and the phase-4 primitive
renames (dev, utils, driver) have done their job — integration/ tests can
now import directly from the real modules.

Rewrite every shim-import in tests/integration/tests/:
  fixtures.authutils → fixtures.auth
  fixtures.alertutils → fixtures.alerts
  fixtures.cloudintegrationsutils → fixtures.cloudintegrations
  fixtures.idputils → fixtures.idp
  fixtures.gatewayutils → fixtures.gateway
  fixtures.utils (get_testdata_file_path) → fixtures.fs

Delete all 8 shim files:
  fixtures/{authutils,alertutils,cloudintegrationsutils,idputils,
  gatewayutils,dev,utils,driver}.py

Nothing in active code (integration tests, e2e fixtures, bootstrap, seeder)
imported fixtures.dev or fixtures.driver, so those had no callers to
sweep — just delete.

500 tests still collect.

* fix(tests/seeder): add python3-dev so psycopg2 can compile in the image

Consolidating seeder deps into pyproject.toml pulled in psycopg2, which
needs Python dev headers (Python.h) to build from source. The apt layer
had gcc + libpq-dev but was missing python3-dev, so \`uv sync --frozen
--no-install-project --no-dev\` failed with "gcc failed with exit code 1"
during the seeder image build.

Add python3-dev to the apt install line; image size bump ~50MB for dev
headers. Alternative would have been swapping psycopg2 for
psycopg2-binary in pyproject.toml, but that'd affect the whole test
project for one Dockerfile concern — wrong scope.

* feat(tests/e2e): re-author 2095 alerts + downtime regression

Three fresh specs split by resource replace the 736-line
legacy monolith at tests/e2e/legacy/alerts/alerts-downtime.spec.ts:

- alerts.spec.ts: rule list CRUD, labels round-trip, test-notification
  pre-state, details/history/AlertNotFound, anomaly (EE-gated, skip on
  community)
- downtime.spec.ts: planned-downtime CRUD round-trip
- cascade-delete.spec.ts: 409 paths on rule/downtime delete when linked

UI-first: Playwright traces capture the BE conversations, so direct
page.request calls are reserved for seeding where the query-builder
setup is incidental to the test, API-contract probes, and cleanup.

* refactor(tests/e2e): group alerts specs under tests/alerts/

* feat(tests/fixtures/auth): apply_license fixture + wire into e2e bootstrap

- Adds a package-scoped apply_license fixture that stubs the Zeus
  /v2/licenses/me mock and POSTs /api/v3/licenses so the BE flips to
  ENTERPRISE. The fixture also PUTs org_onboarding=true because the
  license enables the onboarding flag which would otherwise hijack
  every post-login navigation to a questionnaire.
- Wires apply_license into e2e/bootstrap/setup.py::test_setup and
  ::test_teardown alongside create_user_admin.
- Existing add_license helper stays as-is for integration tests.
- Login fixture now waits for the URL to leave /login instead of a
  pre-license "Hello there" welcome string (the post-login landing
  page varies with license state).
- TC-07 anomaly test no longer skips (license enables the flag) and
  drops the legacy test-notification API contract probe that needs
  seeded metric data (covered by the integration suite).

* chore: cleanup

* chore: remove claude files

* chore(tests/fixtures): drop unused dashboards.py

* chore(tests/e2e): rename playwright outputDir to artifacts/results

* chore(tests/e2e): drop legacy specs, trim alerts.spec.ts to one smoke test

Deletes tests/e2e/legacy/ (five old 2095-replay specs) and the two
sibling alerts suite files (downtime, cascade-delete). alerts.spec.ts
is reduced to a single TC-01 smoke test that loads /alerts and asserts
the tabs render — a fresh minimum to build on.

* docs(contributing): new integration.md + e2e.md at top level

Promotes the two test-contributor docs from docs/contributing/tests/
to docs/contributing/ and rewrites them in the long-form Q&A format
of docs/contributing/go/integration.md (prerequisites → setup →
framework → writing → running → configuring → remember).

Reflects the current state: shared fixtures package at tests/fixtures/,
flat integration suites under tests/integration/tests/, e2e specs
grouped by resource under tests/e2e/tests/<feature>/, apply_license
fixture in the bootstrap, authedPage Playwright fixture, and the
artifacts/{html,json,results} output layout.

* docs(contributing): relocate integration.md + e2e.md to tests/

Moves docs/contributing/go/integration.md -> docs/contributing/tests/integration.md
and docs/contributing/e2e.md -> docs/contributing/tests/e2e.md so the test-
contributor docs live under contributing/tests/. The previous top-level
promotion at docs/contributing/integration.md is removed; go/readme.md
drops the dangling integration link.

* docs(contributing/tests): update integration.md to current repo layout

* ci(tests): fix integrationci paths + add e2eci workflow

integrationci:
- Matrix path was integration/src/<suite> (old layout); current layout
  is integration/tests/<suite>. Renames the matrix key src -> suite and
  fixes the pytest path accordingly.
- Adds auditquerier and rawexportdata to the matrix (new suites).
- Drops bootstrap from the matrix — it's no longer a test suite, just
  the pytest lifecycle entry.

e2eci (new, replaces the broken frontend/-based run-e2e.yaml):
- Label-gated trigger mirroring integrationci: requires safe-to-test +
  safe-to-e2e. Runs on pull_request / pull_request_target.
- Installs Python (uv) and Node (yarn), syncs tests/ deps, installs
  Playwright browsers for the matrix project.
- Brings the stack up via e2e/bootstrap/setup.py::test_setup --with-web
  (build signoz-with-web container once), runs playwright against it,
  tears down in an always-run step.
- Uploads the HTML report + per-test traces as artifacts.
- Matrix starts with chromium only (firefox / webkit can follow).

* ci(tests/e2e): upload entire artifacts/ dir, 5-day retention

* fix(tests/fixtures): apply black formatting to truncate helpers

* fix(tests/pyproject): ignore node_modules and py module in pylint

* ci(tests): drop auditquerier from integrationci matrix for now

* refactor(tests): drop __file__.parents[N] path tricks; use pytestconfig.rootpath

The pytest rootdir is already tests/, so anywhere we were computing
_REPO_ROOT / _TESTS_ROOT / e2e-dir from Path(__file__).resolve().parents[N]
can just use pytestconfig.rootpath (or .parent for the repo root).

- fixtures/signoz.py: DockerImage path → pytestconfig.rootpath.parent
- fixtures/seeder.py: docker-py build path → pytestconfig.rootpath
- e2e/bootstrap/setup.py: .env.local path → pytestconfig.rootpath / e2e
- e2e/bootstrap/run.py: yarn-test cwd → pytestconfig.rootpath / e2e

* chore(tests/e2e): drop bootstrap/run.py

The run.py entrypoint was just setup.py + subprocess('yarn test'); CI
splits those steps anyway (separate provision / test / teardown for
clean artifact capture) and locally the two-step flow is equivalent.
Removing the duplicate entrypoint; docs updated accordingly.

* cleanup(tests): simplify review pass

- fixtures/fs.py: testdata path resolved to tests/testdata after the
  fixture move; integration tests with data-driven parametrize (e.g.
  alerts/02_basic_alert_conditions.py) were all failing with
  FileNotFoundError. Walk to tests/integration/testdata now.
- fixtures/auth.py: extract _login helper so apply_license stops
  duplicating the GET /sessions/context + POST /sessions/email_password
  pair. Add a retry loop on POST /api/v3/licenses so a BE that isn't
  quite ready at bring-up time doesn't fail the fixture.
- seeder/server.py: use FastAPI lifespan to open+close the ClickHouse
  client instead of a lazy module-level global; collapse the verbose
  module docstring.
- fixtures/seeder.py + e2e/bootstrap/setup.py: trim docstrings/comments
  that narrated WHAT the code does — per-repo convention keeps only
  non-obvious WHY.
- .github/workflows/integrationci.yaml: gate the Chrome + chromedriver
  install on matrix.suite == 'callbackauthn' (the only suite that uses
  Selenium). Saves ~30s × 50 jobs on every PR run.
 2026-04-23 10:05:49 +00:00