mirror of
https://github.com/SigNoz/signoz.git
synced 2026-04-25 21:30:26 +01:00
93f5df9185
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
* refactor(tests): hoist pytest project to tests/ root for shared fixtures
Lift pyproject.toml, uv.lock, conftest.py, and fixtures/ up from
tests/integration/ so the pytest project becomes shared infrastructure
rather than integration's private property. A sibling tests/e2e/ can
reuse the same fixture graph (containers, auth, seeding) without
duplicating plugins.
Also:
- Merge tests/integration/src/querier/util.py into tests/fixtures/querier.py
(response assertions and corrupt-metadata generators belong with the
other querier helpers).
- Use --import-mode=importlib + pythonpath=["."] in pyproject so
same-basename tests across src/*/ do not collide at the now-wider
rootdir.
- Broaden python_files to "*/src/**/**.py" so future test trees under
tests/e2e/src/ get discovered.
- Update Makefile py-* targets and integrationci.yaml to cd into tests/
and reference integration/src/... paths.
* feat(tests/e2e): import Playwright suite from signoz-e2e
Relocate the standalone signoz-e2e repository into tests/e2e/ as a
sibling of tests/integration/. The suite still points at remote
staging by default; subsequent commits wire it to the shared pytest
fixture graph so the backend can be provisioned locally.
Excluded from the import: .git, .github (CI migration deferred),
.auth, node_modules, test-results, playwright-report.
* feat(tests/e2e): pytest-driven backend bring-up, seeding, and playwright runner
Wire the Playwright suite into the shared pytest fixture graph so the
backend + its seeded state are provisioned locally instead of pointing
at remote staging.
Python side (owns lifecycle):
- tests/fixtures/dashboards.py — generic create/list/upsert_dashboard
helpers (shared infra; testdata stays per-tree).
- tests/e2e/conftest.py — e2e-scoped pytest fixtures: seed_dashboards
(idempotent upsert from tests/e2e/testdata/dashboards/*.json),
seed_alert_rules (from tests/e2e/testdata/alerts/*.json, via existing
create_alert_rule), seed_e2e_telemetry (fresh traces/logs across a
few synthetic services so /home and Services pages have data).
- tests/e2e/src/bootstrap/setup.py — test_setup depends on the fixture
graph and persists backend coordinates to tests/e2e/.signoz-backend.json;
test_teardown is the --teardown target.
- tests/e2e/src/bootstrap/run.py — test_e2e: one-command entrypoint that
brings up the backend + seeds, then subprocesses yarn test and asserts
Playwright exits 0.
- tests/conftest.py — register fixtures.dashboards plugin.
Playwright side (just reads):
- tests/e2e/global.setup.ts — loads .signoz-backend.json and injects
SIGNOZ_E2E_BASE_URL/USERNAME/PASSWORD. No-op when env is already
populated (staging mode, or pytest-driven runs where env is pre-set).
- playwright.config.ts registers globalSetup.
- package.json gains test:staging; existing scripts unchanged.
Testdata layout: tests/e2e/testdata/{dashboards,alerts,channels}/*.json
— per-tree (integration has its own tests/integration/testdata/).
* docs(tests): describe pytest-master workflow and shared fixture layout
- tests/README.md (new): top-level map of the shared pytest project,
fixture-ownership rule (shared vs per-tree), and common commands.
- tests/e2e/README.md: lead with the one-command pytest run and the
warm-backend dev loop; keep the staging fallback as option 2.
- tests/e2e/CLAUDE.md: updated commands so agent contexts reflect the
pytest-driven lifecycle.
- tests/e2e/.env.example: drop unused SIGNOZ_E2E_ENV_TYPE; note the file
is only needed for staging mode.
* fix(tests/fixtures/signoz.py): anchor Docker build context to repo root
Previously used path="../../" which resolved to the repo root only when
pytest's cwd was tests/integration/. After hoisting the pytest project
to tests/, that same relative path pointed one level above the repo
root and the build failed with:
Cannot locate specified Dockerfile: cmd/enterprise/Dockerfile.with-web.integration
Anchor the build context to an absolute path computed from __file__ so
the fixture works regardless of pytest cwd.
* feat(tests/e2e): alerts-downtime regression suite (platform-pod/issues/2095)
Import the 34-step regression suite originally developed on
platform-pod/issues/2095-frontend. Targets the alerts and planned-downtime
frontend flows after their migration to generated OpenAPI clients and
generated react-query hooks.
- specs/alerts-downtime/: SUITE.md (the stable spec), README.md (scope +
open observations from the original runs), results-schema.md (legacy
per-run artifact shape, retained for context).
- tests/alerts-downtime/alerts-downtime.spec.ts: 881-line Playwright spec
covering 6 flows — alert CRUD/toggle, alert detail 404, planned
downtime CRUD, notification channel routing, anomaly alerts.
Integration with the shared suite:
- Uses baseURL + storageState from tests/e2e/playwright.config.ts (no
separate config). page.goto calls use relative paths; SIGNOZ_E2E_*
env vars from the pytest bootstrap drive auth.
- test.describe.configure({ mode: 'serial' }) at the top of the describe:
the flows mutate shared tenant state, so parallel runs cause cross-
flow interference (documented in the original 2095 config).
- Per-run artifacts (network captures + screenshots) land in
tests/e2e/tests/alerts-downtime/run-spec-<ts>/ by default — gitignored.
Historical per-run artifacts (~7.5MB of screenshots across run-1 through
run-7) are not imported; they lived at e2e/2095/run-*/ on the original
branch and remain there if needed.
* refactor(fixtures/traces): extract insert + truncate helpers
Pull the ClickHouse insert path out of the insert_traces pytest fixture
into a plain module-level function insert_traces_to_clickhouse(conn,
traces), and move the per-table TRUNCATE loop into truncate_traces_tables
(conn, cluster). The fixture becomes a thin wrapper over both — zero
behavioural change.
Lets the HTTP seeder container (tests/fixtures/seeder/) reuse the exact
same insert + truncate code the pytest fixture uses, so the two stay in
sync as the trace schema evolves.
* feat(fixtures/seeder): HTTP seeder container for fine-grained telemetry seeding
Adds a sibling container alongside signoz/clickhouse/postgres that exposes
HTTP endpoints for direct-ClickHouse telemetry seeding, so Playwright
tests can shape per-test data without going through OTel or the SigNoz
ingestion path.
tests/fixtures/seeder/:
- Dockerfile: python:3.13-slim + the shared fixtures/ tree so the
container can import fixtures.traces and reuse the exact insert path
used by pytest.
- server.py: FastAPI app with GET /healthz, POST /telemetry/traces
(accepts a JSON list matching Traces.from_dict input; auto-tags each
inserted row with resource seeder=true), DELETE /telemetry/traces
(truncates all traces tables).
- requirements.txt: fastapi, uvicorn, clickhouse-connect, numpy plus
sqlalchemy/pytest/testcontainers because fixtures/{__init__,types,
traces}.py import them at module load.
tests/fixtures/seeder/__init__.py: pytest fixture (`seeder`, package-
scoped) that builds the image via docker-py (testcontainers DockerImage
had multi-segment dockerfile issues), starts the container on the
shared network wired to ClickHouse via env vars, and waits for
/healthz. Cache key + restore follow the dev.wrap pattern other
fixtures use for --reuse.
tests/.dockerignore: exclude .venv, caches, e2e node_modules, and test
outputs so the build context is small and deterministic.
tests/conftest.py: register fixtures.seeder as a pytest plugin.
Currently traces-only — logs + metrics follow the same pattern.
* feat(tests/e2e): surface seeder_url to Playwright via globalSetup
- bootstrap/setup.py: test_setup now depends on the seeder fixture and
writes seeder_url into .signoz-backend.json alongside base_url.
- bootstrap/run.py: test_e2e exports SIGNOZ_E2E_SEEDER_URL to the
subprocessed yarn test so Playwright specs can reach the seeder
directly in the one-command path.
- global.setup.ts: if .signoz-backend.json carries seeder_url, populate
process.env.SIGNOZ_E2E_SEEDER_URL. Remains optional — staging mode
leaves it unset.
Playwright specs that want per-test telemetry can:
await fetch(process.env.SIGNOZ_E2E_SEEDER_URL + '/telemetry/traces', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify([...])
});
and await a truncate via DELETE on teardown.
* fix(alerts-downtime): capture load-time GETs before navigation
Flow 1 registered cap.mark() AFTER page.goto() and then called
page.waitForResponse(/api/v2/rules) — but against a fast local backend
the GET /api/v2/rules response arrived during page.goto, before the
waiter could register, and the test timed out at 30s.
installCapture's page.on('response') listener runs from before the
navigation, so moving mark() above page.goto() and relying on
dumpSince's 500ms drain is enough. No lost precision.
One site only; the same pattern exists in later flows (via per-action
waitForResponse) and may surface similar races — those are left for a
follow-up once the backend-side 2095 migration lands on main (current
frontend still calls PATCH /api/v1/rules/:id which the spec's assertion
doesn't match anyway).
* refactor(fixtures/logs,metrics): extract insert + truncate helpers
Mirror the traces refactor: pull the ClickHouse insert path out of the
insert_logs / insert_metrics pytest fixtures into plain module-level
functions (insert_logs_to_clickhouse, insert_metrics_to_clickhouse) and
move the per-table TRUNCATE loops into truncate_logs_tables /
truncate_metrics_tables. The fixtures become thin wrappers — zero
behavioural change.
Sets up the seeder container to expose POST/DELETE endpoints for logs
and metrics using the exact same code paths as the pytest fixtures.
* feat(fixtures/seeder): add logs and metrics endpoints
Extend the seeder with POST/DELETE endpoints for logs and metrics,
following the same shape as the existing traces endpoints:
- POST /telemetry/logs accepts a JSON list matching Logs.from_dict;
tags each row's resources with seeder=true.
- POST /telemetry/metrics accepts a JSON list matching Metrics.from_dict;
tags resource_attrs with seeder=true (Metrics.from_dict unpacks
resource_attrs rather than a resources dict).
- DELETE /telemetry/logs, DELETE /telemetry/metrics truncate via the
shared truncate_*_tables helpers.
Requirements gain svix-ksuid because fixtures/logs.py imports KsuidMs
for log id generation.
Verified end-to-end against the warm backend: POST inserted=1 on each
signal, DELETE truncated=true on each.
* refactor(fixtures/seeder): align status codes with HTTP semantics
- POST /telemetry/{traces,logs,metrics}: return 201 Created (kept the
{inserted: N} body so callers can verify the count landed).
- DELETE /telemetry/{traces,logs,metrics}: return 204 No Content with
an empty body.
* refactor(tests/seeder): extract from fixtures/ into top-level package
Move the HTTP seeder (Dockerfile, requirements.txt, server.py) out of
tests/fixtures/seeder/ and into its own tests/seeder/ top-level package.
The pytest fixture that builds and runs the image moves to
tests/fixtures/seeder.py so it sits next to the other container fixtures.
Rationale: the seeder is a standalone containerized Python service, not a
pytest fixture. It ships a Dockerfile, its own requirements.txt, and a
server.py entrypoint — none of which belong under a package whose purpose
is shared pytest code.
Image-side changes:
- Dockerfile now copies seeder/ alongside fixtures/ and launches
seeder.server:app instead of fixtures.seeder.server:app.
- Build context stays tests/ (unchanged), so fixtures.* imports inside
server.py continue to resolve.
Fixture-side changes:
- _TESTS_ROOT computation drops one parent (parents[1] now that the file
is at tests/fixtures/seeder.py, not tests/fixtures/seeder/__init__.py).
- The dockerfile= path passed to docker-py becomes seeder/Dockerfile.
No behavior change; every consumer still imports the seeder fixture as
before and gets the same container.
* refactor(fixtures/keycloak): rename from idp.py to name the concrete tech
The container provider at fixtures/idp.py brought up a Keycloak image. Name
it for what it is so we can use fixtures/idp.py later for API-side IdP
helpers (OIDC/SAML admin flows) without an idp-vs-idputils naming collision.
- fixtures/idp.py → fixtures/keycloak.py (git rename).
- fixtures.idputils updates its one internal import to fixtures.keycloak.
- conftest.py pytest_plugins entry points at the new module.
No caller outside fixtures/ imports fixtures.idp directly, so no shim is
needed. The "idp" fixture name (how tests reference it) is unchanged.
* refactor(fixtures/gateway): drop -utils suffix
The module only held helper functions (no fixtures). Rename to match the
domain and leave a shim at the old path so integration/ import sites keep
working until they are swept in a follow-up.
* fix(fixtures/gatewayutils): silence wildcard-import in deprecation shim
The shim intentionally re-exports via `from fixtures.gateway import *`;
pylint flags the wildcard and every unused-wildcard symbol. Suppress both
in the shim only — the live module has no wildcard.
* refactor(fixtures/auth): merge authutils helpers into auth
Pull the pure-helper functions from authutils.py (create_active_user,
find_user_by_email, find_user_with_roles_by_email, assert_user_has_role,
change_user_role) into auth.py next to the fixtures they complement.
Fixtures remain on top; helpers go below. Drop the module docstring.
Replace authutils.py with a deprecation shim that re-exports from
fixtures.auth so integration/ import sites (9 files) keep working until
they are swept in a follow-up. Suppress the wildcard-import warnings in
the shim only.
* refactor(fixtures/alerts): merge alertutils helpers into alerts
Pull the pure-helper functions from alertutils.py
(collect_webhook_firing_alerts, _verify_alerts_labels,
verify_webhook_alert_expectation, update_rule_channel_name) into alerts.py
next to the fixtures they complement. Fixtures stay on top; helpers go
below.
Replace alertutils.py with a deprecation shim that re-exports from
fixtures.alerts so integration/ import sites keep working until they are
swept in a follow-up.
* refactor(fixtures/cloudintegrations): merge cloudintegrationsutils helpers
Pull the pure-helper functions from cloudintegrationsutils.py
(deprecated_simulate_agent_checkin, setup_create_account_mocks,
simulate_agent_checkin) into cloudintegrations.py next to the fixtures
they complement. Fixtures stay on top; helpers go below.
Replace cloudintegrationsutils.py with a deprecation shim that re-exports
from fixtures.cloudintegrations so integration/ import sites keep working
until they are swept in a follow-up.
* refactor(fixtures/idp): rename idputils to idp now that keycloak owns the container
With the Keycloak container provider at fixtures.keycloak, the fixtures.idp
name is free for what idputils always was — API/browser helpers for OIDC
and SAML admin flows against the IdP container.
- fixtures.idputils → fixtures.idp (git rename).
- conftest.py pytest_plugins swaps fixtures.idputils for fixtures.idp so
the create_saml_client / create_oidc_client fixtures register under the
canonical path.
Replace fixtures.idputils with a deprecation shim re-exporting from
fixtures.idp so integration/ import sites (callbackauthn) keep working
until they are swept in a follow-up.
* refactor(fixtures/reuse): rename from dev to describe what the module is
The module wraps pytest-cache resource reuse/teardown for container
fixtures; "dev" conveyed nothing about its role. Rename to fixtures.reuse
and update the 12 internal callers that imported `from fixtures import
dev, types` to use `reuse` instead.
Replace fixtures.dev with a deprecation shim so any external caller keeps
working until the follow-up sweep.
* refactor(fixtures/time,fs): split utils by responsibility
fixtures.utils only held two time parsers (parse_timestamp, parse_duration)
and one path helper (get_testdata_file_path) — a "utils" grab bag.
- Time parsers move to fixtures.time (utils.py → time.py via git rename).
- get_testdata_file_path moves into fixtures.fs where other filesystem
helpers live.
- Internal callers (alerts, logs, metrics, traces) update to the new paths.
Replace fixtures.utils with a deprecation shim that re-exports all three
functions so integration/ import sites keep working until the follow-up
sweep.
* refactor(fixtures/browser): rename from driver to match peer primitives
fixtures.driver was the Selenium WebDriver fixture — rename the module to
fixtures.browser so it sits next to fixtures.http as a named primitive.
The fixture name inside (driver) stays — that's the Selenium-canonical
term and tests reference it directly.
conftest.py pytest_plugins entry points at the new module. A deprecation
shim at fixtures.driver keeps any external caller working until the
follow-up sweep.
* refactor(tests/seeder): install deps via uv from pyproject, drop requirements.txt
The seeder's requirements.txt duplicated 7 of 10 deps from pyproject.toml
with overlapping version pins — a standing drift risk. The comment on top
of the file admitted the real problem: the seeder image already ships
pytest + testcontainers + sqlalchemy because importing fixtures.traces
walks fixtures/__init__.py and fixtures/types.py. "Don't ship test infra"
was already violated.
- Add fastapi, uvicorn[standard], and py to pyproject.toml dependencies
(the three seeder-only deps that were not yet in pyproject; `py` was a
latent gap since fixtures/types.py uses py.path.local but pytest only
pulls it in transitively).
- Switch the Dockerfile to `uv sync --frozen --no-install-project --no-dev`
so the container env matches local dev exactly (uv.lock is the single
source of truth for versions).
- Move tests/seeder/Dockerfile → tests/Dockerfile.seeder so it lives
alongside the pyproject at the root of the build context.
- Delete tests/seeder/requirements.txt.
The seeder image grows by ~40-50MB (selenium, psycopg2, wiremock now come
along from main deps); accepted as a cost of single source of truth since
the seeder is dev-only infra, not a shipped artifact.
* refactor(tests/integration): flatten src/ into bootstrap/ + tests/
Drop the redundant src/ layer in the integration tree. 'src' carries no
information — the directory IS integration test source. After flatten:
tests/integration/
bootstrap/setup.py was src/bootstrap/setup.py
tests/<suite>/*.py was src/<suite>/*.py (16 suites)
testdata/
Updates:
- Makefile: py-test-setup/py-test-teardown/py-test target paths.
- tests/README.md: layout diagram + command examples.
- tests/pyproject.toml: python_files glob now matches basenames
explicitly — "[0-9][0-9]_*.py" for NN-prefixed suite files plus
"setup.py" and "run.py" for bootstrap entrypoints. The old "*/src/.."
glob stopped matching anything here and would have caused pytest to
try collecting seeder/server.py as a test.
* refactor(tests/e2e): flatten src/ into bootstrap/
Drop the e2e/src/ wrapper — the only Python content under it was
bootstrap/, which is now a direct child of e2e/. Keeps integration and
e2e symmetric (both have bootstrap/, tests/, testdata/ as peers).
Also delete bootstrap/__init__.py on both integration and e2e sides.
With --import-mode=importlib, pytest walks up from each .py file to find
the highest __init__.py-containing dir and uses that as the package root.
Without integration/__init__.py or e2e/__init__.py above bootstrap/, both
setup.py files resolved to the same dotted name `bootstrap.setup`, causing
a sys.modules collision that silently dropped test_telemetry_databases_exist
from integration's bootstrap. With no __init__.py anywhere, pytest treats
each setup.py as a standalone module via spec_from_file_location and both
are collected cleanly.
Updates tests/README.md, tests/e2e/README.md, and tests/e2e/CLAUDE.md path
references from e2e/src/bootstrap/ to e2e/bootstrap/.
* refactor(tests/e2e): drop specs/ + strip // spec: back-pointers
specs/ held markdown test plans that mirrored tests/ 1:1 as pre-code
scratch. Once a test exists, the plan is stale the moment the test
diverges — they're AI-planner output, not source of truth. Keep the
workflow alive by .gitignore-ing specs/ (the planner agent can still
write locally) but stop shipping stale plans in the repo.
Strip the `// spec: specs/...` and `// seed: tests/seed.spec.ts` header
comments from 5 .spec.ts files. The spec pointer is dead; the seed
pointer was convention-only — Playwright collects regardless.
* docs(contributing/tests): move e2e/integration guides out of test dirs
Pull the e2e contributor guide out of tests/e2e/CLAUDE.md (which read
like a full agent-workflow reference doc) and into
docs/contributing/tests/e2e.md alongside the existing development / go
guides.
- Delete tests/e2e/CLAUDE.md; its content (layout, commands, role tags,
locator priority, Playwright agent workflow) lives in the new e2e.md
with references to the now-.gitignore'd specs/ dir removed.
- Add docs/contributing/tests/integration.md — short guide covering
layout, runner commands, filename conventions, and the flow for
adding a new suite (there was no contributor doc for this before).
- Trim tests/e2e/README.md to quick-start + commands; link out to the
full guide. Readers who just want to run tests get the 5 commands
they need; anything deeper is one hop away.
* chore(tests/e2e): drop examples/example-test-plan.md
Init-agents boilerplate. Fresh planner agents don't need a checked-in
template; they can write to the .gitignore'd specs/ scratch dir.
tests/integration/.qodo/ was also removed (untracked, empty; .qodo is
already in the root .gitignore).
* refactor(tests/seeder): use fixtures.logger.setup_logger
Drop the one-off logging.basicConfig + logging.getLogger("seeder") in
favor of the shared setup_logger helper that every fixtures/*.py already
uses. Keeps log format consistent across pytest runs and the seeder
container.
fixtures.logger ships into the image via the existing COPY fixtures step
in Dockerfile.seeder — no build change needed.
* fix(tests/e2e): correct e2e_dir path after src/ flatten
After phase 2 (flatten tests/e2e/src/ into tests/e2e/), the run.py file
sits one level closer to the e2e root. parents[2] now resolves to tests/
instead of tests/e2e/, so yarn test would subprocess from the wrong cwd.
parents[1] is the correct index now.
* fix(tests/e2e): correct endpoint-file path in setup.py after src/ flatten
Same class of stale-path bug as the run.py fix: after the e2e/src/
flatten, setup.py sits one level closer to the e2e root. parents[2] now
lands at tests/ instead of tests/e2e/, so .signoz-backend.json would be
written to tests/.signoz-backend.json and the Playwright global.setup.ts
(which expects tests/e2e/.signoz-backend.json) wouldn't find it.
parents[1] is correct.
* refactor(tests/e2e): drop pre-seed fixtures; each spec owns its data
The seeder (tests/seeder/) was built so specs can POST telemetry
per-test. Global pre-seeding via tests/e2e/conftest.py (seed_dashboards,
seed_alert_rules, seed_e2e_telemetry) is the exact anti-pattern that
setup obsoletes — shared state across specs, order-dependent runs, no
reset between tests.
- Delete tests/e2e/conftest.py (3 fixtures, all pre-seed).
- Delete tests/e2e/testdata/dashboards/apm-metrics.json — its only
consumer was seed_dashboards. tests/e2e/testdata/ now empty and gone.
- Drop seed_dashboards, seed_alert_rules, seed_e2e_telemetry params
from bootstrap/setup.py::test_setup and bootstrap/run.py::test_e2e.
test_teardown never depended on them.
- Refresh the module docstrings on both bootstrap tests to reflect the
new model (backend + seeder up; specs seed themselves).
- Update tests/README.md and docs/contributing/tests/e2e.md: remove the
testdata/ + conftest.py references, document the per-spec seeding
rule (telemetry via seeder endpoints, dashboards/alerts via SigNoz
REST API from the spec).
Known breakage: tests/e2e/tests/dashboards/dashboards-list.spec.ts
expects at least one dashboard to exist. With seed_dashboards gone, it
will fail until that spec is updated to create its own dashboard via
the SigNoz API in test.beforeAll. Followup.
* refactor(tests/e2e): relocate auth helper into fixtures/; expose authedPage
Rename tests/e2e/utils/login.util.ts → tests/e2e/fixtures/auth.ts and
drop the (now-empty) utils/ dir. "Fixtures" is the unit of per-test
shared setup on both the Python and TS sides of this project — naming
them consistently across trees makes the parallel obvious.
fixtures/auth.ts now exports three things:
- `test` — Playwright test extended with an authedPage fixture. New
specs can request `authedPage` as a param and skip the
`beforeEach(() => ensureLoggedIn(page))` boilerplate entirely.
- `expect` — re-exported from @playwright/test so callers have one
import.
- `ensureLoggedIn(page)` — the underlying helper, still exported for
specs that want per-call control.
Update the 4 specs that imported from utils/login.util to point at the
new path; no behavior change in those specs (they keep calling
ensureLoggedIn in beforeEach). Refactoring them to use authedPage can
happen spec-by-spec later.
Also update the path example in .cursorrules so AI-generated snippets
reach for the new import path.
* refactor(tests/e2e): emit .env.local instead of .signoz-backend.json
The old flow (pytest writes JSON → global.setup.ts loads it → exports
env vars) was doing what dotenv already does. Collapse to the native
pattern:
- bootstrap/setup.py writes tests/e2e/.env.local with the four coords
(BASE_URL, USERNAME, PASSWORD, SEEDER_URL). File header marks it as
generated.
- playwright.config.ts loads .env first, then .env.local with
override=true. User-provided defaults stay in .env; generated values
win when present.
- Delete tests/e2e/global.setup.ts (36 lines gone) and its globalSetup
reference in playwright.config.ts.
Subprocess-injected env (run.py shelling out to yarn test) still wins
because dotenv doesn't overwrite already-set process.env keys.
Rename the test-only override env var SIGNOZ_E2E_ENDPOINT_FILE →
SIGNOZ_E2E_ENV_FILE for accuracy. Update .env.example, .gitignore (drop
.signoz-backend.json, keep .env.local with its explanatory comment),
tests/README.md, docs/contributing/tests/e2e.md.
* refactor(tests/e2e/alerts-downtime): drop custom network + screenshot capture
The spec wrapped every /api/ response in a bespoke installCapture(), wrote
hand-named JSON files per call (01_step1.1_GET_rules.json, ...), and took
step-by-step screenshots — all going into run-spec-<ts>/ next to the spec
(gitignored).
Playwright already records equivalent data via `trace` (network bodies,
screenshots per step, DOM snapshots, console — viewable via
`playwright show-trace`). The capture infra was duplicating that for the
one-shot 2095 regression audit; no downstream consumer reads the JSON or
PNG artifacts now.
- Remove installCapture, shot, RUN_DIR/NET_DIR/SHOT_DIR, fs/path imports.
- Strip cap.mark()/cap.dumpSince()/shot() calls throughout the 7 flows.
- Collapse the block-scopes that only existed to bound mark variables.
- Drop the "Artifacts" paragraph from the file's top-of-file comment.
- Remove the `tests/alerts-downtime/run-spec-*/` entry from .gitignore.
Spec drops from 885 lines to 736 (≈17% smaller). All 7 flows + their
assertions are unchanged. For debug access, rely on
`trace: 'on-first-retry'` (already set in playwright.config.ts) + `yarn
show-trace`.
* refactor(tests/e2e): move alerts-downtime.spec.ts into alerts/
The spec lives mostly in the alerts domain (6 of 7 flows), with the
planned-downtime CRUD (Flow 4) and cascade-delete (Flow 5) as
cross-feature collateral. The standalone alerts-downtime/ dir was
compound-named, breaking the one-feature-per-dir pattern every other
dir under tests/ follows, and duplicating the spec's own filename.
Move to tests/alerts/alerts-downtime.spec.ts. Empty alerts-downtime/
dir removed.
* refactor(tests/e2e): consolidate Playwright output under artifacts/
All Playwright outputs now land under a single tests/e2e/artifacts/ dir
so CI can archive it in one command (tar / zip / upload-artifact). Each
piece was writing to its own sibling of tests/e2e/ before.
playwright.config.ts:
- outputDir: 'artifacts/test-results' — per-test traces, screenshots,
videos (was default test-results/).
- HTML reporter → 'artifacts/html-report' (was default
playwright-report/); open: 'never' so CI doesn't spawn a browser on
report generation.
- JSON reporter → 'artifacts/results.json' (was
'test-results/results.json').
package.json: `yarn report` now points playwright show-report at the new
HTML folder.
Ignore updates — replace the two old paths with /artifacts/ in
tests/e2e/.gitignore, tests/e2e/.prettierignore, and tests/.dockerignore
(seeder image build context).
.cursorrules: update the `cat test-results/results.json` example to the
new path so AI-generated snippets reach for the right file.
Delete the empty test-results/ and playwright-report/ dirs that prior
runs left behind.
* refactor(tests/e2e): one artifacts/ subdir per reporter
Within artifacts/, give each reporter its own named subdir so the layout
tells you what wrote what:
artifacts/
html/ # HTML reporter (was artifacts/html-report)
json/results.json # JSON reporter (was artifacts/results.json)
test-results/ # outputDir — per-test traces/screenshots/videos
`yarn report` and the .cursorrules cat example point at the new paths.
* refactor(tests/e2e): drop SIGNOZ_USER_ROLE env filter and @admin/@editor/@viewer tags
The filter claimed to be role-based but only grep'd by tag — the actual
browser session is always admin (bootstrap creates one admin, auth.setup.ts
saves one storageState, every project uses it). Tagging tests `@viewer`
didn't mean they ran as a viewer; it just meant they'd be in the subset
selected when SIGNOZ_USER_ROLE=Viewer. Superset semantics (admin sees
everything) meant the filter was at best a narrower test selection and
at worst a misleading assertion of role coverage.
Gone:
- getRoleGrepPattern() + grep: line in playwright.config.ts.
- The dedicated setup project's grep override (no filter to override).
- SIGNOZ_USER_ROLE entries in .env.example, README, docs/contributing.
- The "Role-Based Testing" section + all role-tagging guidance and
example snippets in .cursorrules.
- All `{ tag: '@viewer' | '@editor' | '@admin' }` annotations on the 90
affected test sites across 5 spec files (single-line and multi-line
forms). ~90 annotations gone.
For ad-hoc selection, `yarn test --grep <pattern>` still works on
Playwright's normal grep (test titles/paths).
Real role-based coverage (separate users + storageStates per role) is a
different problem — not pretending this was it.
* chore(tests/e2e): drop .cursorrules
* refactor(tests/e2e): move auth from project-level storageState to per-suite fixture
Replaced auth.setup.ts + globally-mounted storageState with a test-scoped
authedPage fixture in tests/e2e/fixtures/auth.ts. Each suite controls its
own identity via `test.use({ user: ... })`; specs that need to run
unauthenticated just request the stock `page` fixture instead.
fixtures/auth.ts:
- Declares `user` as a test option, defaulting to ADMIN (creds from
.env.local / .env).
- authedPage resolves to a Page whose context has storageState mounted
for that user. First request per (user, worker) triggers one login
and writes a per-user storageState file under .auth/; subsequent
requests reuse it via a Promise-valued cache.
- Exposes `User` type and `ADMIN` constant so future suites can declare
additional users (EDITOR, VIEWER) as credentials become available.
playwright.config.ts:
- Drop authFile constant, `setup` project, storageState + dependencies
on each browser project.
tests/auth.setup.ts:
- Deleted. Login logic now lives inside fixtures/auth.ts's login() helper,
called on demand by the fixture rather than upfront for the whole run.
Spec migration (6 files):
- Import `test, expect` from ../fixtures/auth (or ../../fixtures/auth)
instead of @playwright/test.
- Drop `ensureLoggedIn` imports and `await ensureLoggedIn(page)` calls.
- Swap `{ page }` → `{ authedPage: page }` in test and beforeEach
destructures (local var stays `page` via aliasing so test bodies need
no further changes).
Cost: N logins per run, where N = unique users × workers (= 1 × 2–4
today, vs the old 1 globally). Tradeoff for explicit per-suite control.
Specs that need unauth later just use `async ({ page }) => ...` — the
fixture isn't invoked, so no login fires.
291 tests still list (previously 292: the old auth.setup.ts counted as
one fake "test"; it's gone now).
* refactor(tests/e2e): cache auth storageState in memory, drop .auth/ dir
The fixture was writing each user's storageState to .auth/<user>.json and
then handing Playwright the file path. But Playwright's
browser.newContext({ storageState }) accepts the object form too —
ctx.storageState() without a path arg returns the cookies+origins
inline.
Keeping the cache in memory means no filesystem roundtrip per login, no
.auth/ dir to maintain, no stale JSON persisting across runs, and no
gitignore entry for it. Each worker's Map holds one Promise<StorageState>
per unique user, resolved on first login and reused thereafter.
Drop the .auth/ entry from tests/e2e/.gitignore; delete the (now unused)
on-disk .auth/ dir.
* chore(tests/e2e): drop seed.spec.ts
* chore(tests/e2e): drop unused README.md and .mcp.json
* refactor(tests/e2e): move existing specs to legacy/ pending fresh rewrite
Park the 5 current spec files under tests/e2e/legacy/ while fresh specs
get written in tests/e2e/tests/ against the new conventions (TC-NN
titles, authedPage fixture, minimal direct-fetch). Playwright's testDir
stays pointed at ./tests — `yarn test` now finds 0 tests until the
first fresh spec lands. legacy/ is preserved for reference but not
collected by default.
Add a .gitkeep under tests/ so the empty dir survives in git between
the move and the first new spec.
Running legacy on demand:
npx playwright test --config tests/e2e/playwright.config.ts \
--project chromium legacy/<spec>.ts
(or temporarily point testDir at ./legacy in the config). No yarn
script wired — legacy is expected to rot as fresh specs replace it.
* refactor(tests): drop -utils deprecation shims; import from canonical modules
The shims we introduced during the phase-3 merges (authutils, alertutils,
cloudintegrationsutils, idputils, gatewayutils) and the phase-4 primitive
renames (dev, utils, driver) have done their job — integration/ tests can
now import directly from the real modules.
Rewrite every shim-import in tests/integration/tests/:
fixtures.authutils → fixtures.auth
fixtures.alertutils → fixtures.alerts
fixtures.cloudintegrationsutils → fixtures.cloudintegrations
fixtures.idputils → fixtures.idp
fixtures.gatewayutils → fixtures.gateway
fixtures.utils (get_testdata_file_path) → fixtures.fs
Delete all 8 shim files:
fixtures/{authutils,alertutils,cloudintegrationsutils,idputils,
gatewayutils,dev,utils,driver}.py
Nothing in active code (integration tests, e2e fixtures, bootstrap, seeder)
imported fixtures.dev or fixtures.driver, so those had no callers to
sweep — just delete.
500 tests still collect.
* fix(tests/seeder): add python3-dev so psycopg2 can compile in the image
Consolidating seeder deps into pyproject.toml pulled in psycopg2, which
needs Python dev headers (Python.h) to build from source. The apt layer
had gcc + libpq-dev but was missing python3-dev, so \`uv sync --frozen
--no-install-project --no-dev\` failed with "gcc failed with exit code 1"
during the seeder image build.
Add python3-dev to the apt install line; image size bump ~50MB for dev
headers. Alternative would have been swapping psycopg2 for
psycopg2-binary in pyproject.toml, but that'd affect the whole test
project for one Dockerfile concern — wrong scope.
* feat(tests/e2e): re-author 2095 alerts + downtime regression
Three fresh specs split by resource replace the 736-line
legacy monolith at tests/e2e/legacy/alerts/alerts-downtime.spec.ts:
- alerts.spec.ts: rule list CRUD, labels round-trip, test-notification
pre-state, details/history/AlertNotFound, anomaly (EE-gated, skip on
community)
- downtime.spec.ts: planned-downtime CRUD round-trip
- cascade-delete.spec.ts: 409 paths on rule/downtime delete when linked
UI-first: Playwright traces capture the BE conversations, so direct
page.request calls are reserved for seeding where the query-builder
setup is incidental to the test, API-contract probes, and cleanup.
* refactor(tests/e2e): group alerts specs under tests/alerts/
* feat(tests/fixtures/auth): apply_license fixture + wire into e2e bootstrap
- Adds a package-scoped apply_license fixture that stubs the Zeus
/v2/licenses/me mock and POSTs /api/v3/licenses so the BE flips to
ENTERPRISE. The fixture also PUTs org_onboarding=true because the
license enables the onboarding flag which would otherwise hijack
every post-login navigation to a questionnaire.
- Wires apply_license into e2e/bootstrap/setup.py::test_setup and
::test_teardown alongside create_user_admin.
- Existing add_license helper stays as-is for integration tests.
- Login fixture now waits for the URL to leave /login instead of a
pre-license "Hello there" welcome string (the post-login landing
page varies with license state).
- TC-07 anomaly test no longer skips (license enables the flag) and
drops the legacy test-notification API contract probe that needs
seeded metric data (covered by the integration suite).
* chore: cleanup
* chore: remove claude files
* chore(tests/fixtures): drop unused dashboards.py
* chore(tests/e2e): rename playwright outputDir to artifacts/results
* chore(tests/e2e): drop legacy specs, trim alerts.spec.ts to one smoke test
Deletes tests/e2e/legacy/ (five old 2095-replay specs) and the two
sibling alerts suite files (downtime, cascade-delete). alerts.spec.ts
is reduced to a single TC-01 smoke test that loads /alerts and asserts
the tabs render — a fresh minimum to build on.
* docs(contributing): new integration.md + e2e.md at top level
Promotes the two test-contributor docs from docs/contributing/tests/
to docs/contributing/ and rewrites them in the long-form Q&A format
of docs/contributing/go/integration.md (prerequisites → setup →
framework → writing → running → configuring → remember).
Reflects the current state: shared fixtures package at tests/fixtures/,
flat integration suites under tests/integration/tests/, e2e specs
grouped by resource under tests/e2e/tests/<feature>/, apply_license
fixture in the bootstrap, authedPage Playwright fixture, and the
artifacts/{html,json,results} output layout.
* docs(contributing): relocate integration.md + e2e.md to tests/
Moves docs/contributing/go/integration.md -> docs/contributing/tests/integration.md
and docs/contributing/e2e.md -> docs/contributing/tests/e2e.md so the test-
contributor docs live under contributing/tests/. The previous top-level
promotion at docs/contributing/integration.md is removed; go/readme.md
drops the dangling integration link.
* docs(contributing/tests): update integration.md to current repo layout
* ci(tests): fix integrationci paths + add e2eci workflow
integrationci:
- Matrix path was integration/src/<suite> (old layout); current layout
is integration/tests/<suite>. Renames the matrix key src -> suite and
fixes the pytest path accordingly.
- Adds auditquerier and rawexportdata to the matrix (new suites).
- Drops bootstrap from the matrix — it's no longer a test suite, just
the pytest lifecycle entry.
e2eci (new, replaces the broken frontend/-based run-e2e.yaml):
- Label-gated trigger mirroring integrationci: requires safe-to-test +
safe-to-e2e. Runs on pull_request / pull_request_target.
- Installs Python (uv) and Node (yarn), syncs tests/ deps, installs
Playwright browsers for the matrix project.
- Brings the stack up via e2e/bootstrap/setup.py::test_setup --with-web
(build signoz-with-web container once), runs playwright against it,
tears down in an always-run step.
- Uploads the HTML report + per-test traces as artifacts.
- Matrix starts with chromium only (firefox / webkit can follow).
* ci(tests/e2e): upload entire artifacts/ dir, 5-day retention
* fix(tests/fixtures): apply black formatting to truncate helpers
* fix(tests/pyproject): ignore node_modules and py module in pylint
* ci(tests): drop auditquerier from integrationci matrix for now
* refactor(tests): drop __file__.parents[N] path tricks; use pytestconfig.rootpath
The pytest rootdir is already tests/, so anywhere we were computing
_REPO_ROOT / _TESTS_ROOT / e2e-dir from Path(__file__).resolve().parents[N]
can just use pytestconfig.rootpath (or .parent for the repo root).
- fixtures/signoz.py: DockerImage path → pytestconfig.rootpath.parent
- fixtures/seeder.py: docker-py build path → pytestconfig.rootpath
- e2e/bootstrap/setup.py: .env.local path → pytestconfig.rootpath / e2e
- e2e/bootstrap/run.py: yarn-test cwd → pytestconfig.rootpath / e2e
* chore(tests/e2e): drop bootstrap/run.py
The run.py entrypoint was just setup.py + subprocess('yarn test'); CI
splits those steps anyway (separate provision / test / teardown for
clean artifact capture) and locally the two-step flow is equivalent.
Removing the duplicate entrypoint; docs updated accordingly.
* cleanup(tests): simplify review pass
- fixtures/fs.py: testdata path resolved to tests/testdata after the
fixture move; integration tests with data-driven parametrize (e.g.
alerts/02_basic_alert_conditions.py) were all failing with
FileNotFoundError. Walk to tests/integration/testdata now.
- fixtures/auth.py: extract _login helper so apply_license stops
duplicating the GET /sessions/context + POST /sessions/email_password
pair. Add a retry loop on POST /api/v3/licenses so a BE that isn't
quite ready at bring-up time doesn't fail the fixture.
- seeder/server.py: use FastAPI lifespan to open+close the ClickHouse
client instead of a lazy module-level global; collapse the verbose
module docstring.
- fixtures/seeder.py + e2e/bootstrap/setup.py: trim docstrings/comments
that narrated WHAT the code does — per-repo convention keeps only
non-obvious WHY.
- .github/workflows/integrationci.yaml: gate the Chrome + chromedriver
install on matrix.suite == 'callbackauthn' (the only suite that uses
Selenium). Saves ~30s × 50 jobs on every PR run.