signoz/docs/contributing/tests/e2e.md

E2E Tests

SigNoz uses end-to-end tests to verify the frontend works correctly against a real backend. These tests use Playwright to drive a real browser against a containerized SigNoz stack that pytest brings up — the same fixture graph integration tests use, with an extra HTTP seeder container for per-spec telemetry seeding.

How to set up the E2E test environment?

Prerequisites

Before running E2E tests, ensure you have the following installed:

• Python 3.13+
• uv
• Docker (for containerized services)
• Node 18+ and Yarn

Initial Setup

1. Install Python deps for the shared tests project:

cd tests
uv sync

2. Install Node deps and Playwright browsers:

cd e2e
yarn install
yarn install:browsers   # one-time Playwright browser install

Starting the Test Environment

To spin up the backend stack (SigNoz, ClickHouse, Postgres, ClickHouse Keeper, Zeus mock, gateway mock, seeder, migrator-with-web) and keep it running:

cd tests
uv run pytest --basetemp=./tmp/ -vv --reuse --rebuild --with-web \
  e2e/bootstrap/setup.py::test_setup

This command will:

• Bring up all containers via pytest fixtures
• Register the admin user (admin@integration.test / password123Z$)
• Apply the enterprise license (via a WireMock stub of Zeus) and dismiss the org-onboarding prompt so specs can navigate directly to feature pages
• Start the HTTP seeder container (tests/seeder/ — exposing /telemetry/{traces,logs,metrics} POST + DELETE)
• Write backend coordinates to tests/e2e/.env.local (loaded by playwright.config.ts via dotenv)
• Keep containers running via the --reuse flag
• Rebuild the SigNoz container from the current sources via the --rebuild flag

The --with-web flag builds the frontend into the SigNoz container — required for E2E. The build takes ~4 mins on a cold start; later builds are incremental.

Rebuilding After Source Changes

The --with-web image bakes the built frontend in, so neither backend nor frontend changes are picked up while --reuse keeps the container running. --rebuild fixes that for both: it kills the SigNoz container, rebuilds the image incrementally (go build cache + pnpm store — a frontend-only change rebuilds in about a minute), and starts a fresh one while databases, mocks, migrations, and the seeder stay reused. The setup command above passes it, so the iteration loop is: change code → re-run the setup command → re-run your specs. --rebuild requires --reuse and cannot be combined with --teardown or --clean.

Stopping the Test Environment

When you're done writing E2E tests, clean up the environment:

cd tests
uv run pytest --basetemp=./tmp/ -vv --teardown \
  e2e/bootstrap/setup.py::test_teardown

Understanding the E2E Test Framework

Playwright drives a real browser (Chromium / Firefox / WebKit) against the running SigNoz frontend. The backend is brought up by the same pytest fixture graph integration tests use, so both suites share one source of truth for container lifecycle, license seeding, and test-user accounts.

• Why Playwright? First-class TypeScript support, network interception, automatic wait-for-visibility, built-in trace viewer that captures every request/response the UI triggers — so specs rarely need separate API probes alongside UI clicks.
• Why pytest for lifecycle? The integration suite already owns container bring-up. Reusing it keeps the E2E stack exactly in sync with the integration stack and avoids a parallel lifecycle framework.
• Why a separate seeder container? Per-spec telemetry seeding (traces / logs / metrics) needs a thin HTTP wrapper around the ClickHouse insert helpers so a browser spec can POST from inside the test. The seeder lives at tests/seeder/, is built from tests/Dockerfile.seeder, and reuses the same fixtures/{traces,logs,metrics}.py as integration tests.

tests/
├── fixtures/                  # shared with integration (see integration.md)
├── integration/               # pytest integration suite
├── seeder/                    # standalone HTTP seeder container
│   ├── __init__.py
│   ├── Dockerfile
│   └── server.py              # FastAPI app wrapping fixtures.{traces,logs,metrics}
└── e2e/
    ├── package.json
    ├── playwright.config.ts   # loads .env + .env.local via dotenv
    ├── .env.example           # staging-mode template
    ├── .env.local             # generated by bootstrap/setup.py (gitignored)
    ├── bootstrap/
    │   └── setup.py           # test_setup / test_teardown — pytest lifecycle
    ├── fixtures/              # Playwright test fixtures (test.extend) only
    │   └── auth.ts
    ├── helpers/               # function helpers + the constants they share with tests
    │   ├── auth.ts
    │   └── dashboards.ts
    ├── testdata/              # static data files (JSON) used by helpers and tests
    │   └── apm-metrics.json   # (example)
    ├── tests/                 # Playwright .spec.ts files, one dir per feature area
    │   └── alerts/
    │       └── alerts.spec.ts # (example)
    └── artifacts/             # per-run output (gitignored)
        ├── html/              # HTML reporter output
        ├── json/              # JSON reporter output
        └── results/           # per-test traces / screenshots / videos on failure

fixtures/ vs helpers/ — what goes where

These two folders look similar but mean different things:

• fixtures/ holds Playwright test fixtures (created via test.extend({...})). By the canonical definition, a fixture is "a consistent, predefined set of data, objects, or environmental conditions used to ensure tests run in a stable state" — i.e. setup/teardown that runs automatically around each test or worker. auth.ts matches: it extends Playwright's test with an authedPage that's logged-in before every test runs and torn down after. If the only thing in this folder ever is auth.ts, that's fine — fixtures are a deliberately small surface.
• helpers/ holds plain function helpers that you call explicitly from a test or hook — they don't extend Playwright's test. This covers both behaviour helpers (e.g. gotoDashboardsList(page)) and the constants those helpers and the tests both refer to (e.g. SEARCH_PLACEHOLDER). Constants live next to the helpers that use them so a single import line in a test covers both.
• testdata/ holds static data files (typically JSON / YAML) consumed by the helpers — for example, apm-metrics.json, a real dashboard payload uploaded through the UI by an importer helper.

Rule of thumb: if it's a test.extend fixture, put it in fixtures/. If it's a function you call explicitly (or a constant the function uses), put it in helpers/. If it's a static file the helpers read, put it in testdata/.

Extended fixtures

For features needing complex setup (API-seeded data, ruler evaluation waits, cleanup), create domain-specific fixtures that extend auth. Group them in fixtures/<domain>/.

Fixture scopes:

• test scope — fresh data per test. Use for mutations (edit, delete, rename).
• worker scope — shared across tests in one worker. Use for read-only data. Worker scope pays the setup cost once per worker instead of once per test.

The alerts pattern (fixtures/alerts/) demonstrates extending fixtures:

fixtures/alerts/
├── alert-rules.ts   # extends auth — worker-scoped rule list + test-scoped factory
└── alert-history.ts # extends alert-rules — adds history fixtures (waits on ruler)

Specs import from the fixture they need:

// List tests — just need rules, no history
import { test, expect } from '../../../fixtures/alerts/alert-rules';

// History tests — need history rows from ruler evaluation
import { test, expect } from '../../../fixtures/alerts/alert-history';

When creating new fixtures:

1. Identify scope — Will tests mutate the data? If yes, test-scoped. If read-only, worker-scoped.
2. Group by domain — Put fixtures in fixtures/<domain>/. Helpers in helpers/<domain>/.
3. Extend existing fixtures — Chain from auth or another fixture to inherit its setup.
4. Handle timeouts — Worker-scoped fixtures that wait on backend processing need explicit timeouts.
5. Clean up — Always delete seeded data in the fixture teardown (after use()).
6. Extract logic into functions — Keep the test.extend() block lean; move setup/teardown logic to named functions so the extend block reads as a manifest of "what fixtures exist."

Each spec follows these principles:

1. Directory per feature: tests/e2e/tests/<feature>/*.spec.ts. Cross-resource junction concerns (e.g. cascade-delete) go in their own file, not packed into one giant spec.
2. Test titles use TC-NN: test('TC-01 alerts page — tabs render', ...). Preserves ordering at a glance and maps to external coverage tracking.
3. UI-first: drive flows through the UI. Playwright traces capture every BE request/response the UI triggers, so asserting on UI outcomes implicitly validates BE contracts. Reach for direct page.request.* only when the test's purpose is asserting a response contract (use page.waitForResponse on a UI click) or when a specific UI step is structurally flaky (e.g. Ant DatePicker calendar-cell indices) — and even then try UI first.
4. Self-contained state: each spec seeds its own data and cleans up at suite teardown. The pytest harness creates a fresh stack with zero dashboards / alerts / etc. — never assume pre-existing data. Two patterns work:
   • Per-test seed + cleanup in try / finally — small specs where each test owns its data.
   • Suite-level seed + afterAll teardown — preferred for larger specs. Each createDashboard(...) call adds the resulting ID to a module-level Set<string>, and one test.afterAll(...) deletes everything in the set. See tests/e2e/tests/dashboards/list.spec.ts for the full pattern. test.beforeAll / test.afterAll cannot use authedPage directly (it's test-scoped); use newAdminContext(browser) from helpers/auth.ts instead — it performs one fresh login per suite hook.
5. Seed via API when the UI flow is multi-step or brittle. The frontend stores its JWT in localStorage under AUTH_TOKEN; page.request.* inherits the auth fixture's storage state. A typical pattern:

   const token = await page.evaluate(
     () => (globalThis as any).localStorage.getItem('AUTH_TOKEN') || '',
   );
   await page.request.post('/api/v1/dashboards', {
     data: { title: 'my-name', uploadedGrafana: false },
     headers: { Authorization: `Bearer ${token}` },
   });
   

   This is faster and more reliable than a multi-step UI seed. Reach for the UI flow only when the test's purpose is asserting that flow.
6. Reusable static data lives in tests/e2e/testdata/. For example, apm-metrics.json is a real dashboard payload that importApmMetricsDashboardViaUI (in helpers/dashboards.ts) uploads through the actual Import JSON UI flow to seed a richly-tagged dashboard for search/list tests.

How to write an E2E test?

Create a new file tests/e2e/tests/alerts/smoke.spec.ts:

import { test, expect } from '../../fixtures/auth';

test('TC-01 alerts page — tabs render', async ({ authedPage: page }) => {
  await page.goto('/alerts');
  await expect(page.getByRole('tab', { name: /alert rules/i })).toBeVisible();
  await expect(page.getByRole('tab', { name: /configuration/i })).toBeVisible();
});

The authedPage fixture (from tests/e2e/fixtures/auth.ts) gives you a Page whose browser context is already authenticated as the admin user. First use per worker triggers one login; the resulting storageState is held in memory and reused for later requests.

To run just this test (assuming the stack is up via test_setup):

cd tests/e2e
npx playwright test tests/alerts/smoke.spec.ts --project=chromium

Here's a more comprehensive example that exercises a CRUD flow via the UI:

import { test, expect } from '../../fixtures/auth';

test.describe.configure({ mode: 'serial' });

test('TC-02 alerts list — create, toggle, delete', async ({ authedPage: page }) => {
  await page.goto('/alerts?tab=AlertRules');
  const name = 'smoke-rule';

  // Seed via UI — click "New Alert", fill form, save.
  await page.getByRole('button', { name: /new alert/i }).click();
  await page.getByTestId('alert-name-input').fill(name);
  // ... fill metric / threshold / save ...

  // Find the row and exercise the action menu.
  const row = page.locator('tr', { hasText: name });
  await expect(row).toBeVisible();
  await row.locator('[data-testid="alert-actions"] button').first().click();

  // waitForResponse captures the network call the UI triggers — no parallel fetch needed.
  const patchWait = page.waitForResponse(
    (r) => r.url().includes('/rules/') && r.request().method() === 'PATCH',
  );
  await page.getByRole('menuitem').filter({ hasText: /^disable$/i }).click();
  await patchWait;
  await expect(row).toContainText(/disabled/i);
});

Locator priority

1. getByTestId('...') — preferred when the source exposes one. Stable, app-author-provided handle that survives copy-edits.
2. getByRole('button', { name: 'Submit' })
3. getByLabel('Email')
4. getByPlaceholder('...')
5. getByText('...')
6. locator('.ant-select') — last resort (Ant Design dropdowns often have no semantic alternative)

Agents

Three Claude agents in .claude/agents/ accelerate writing and maintaining E2E specs:

• playwright-test-planner — explores a feature in a real browser plus the local frontend source and writes a test plan as a scratch markdown file (under tests/e2e/specs/, which is gitignored — plans are working artifacts for the generator, not committed docs).
• playwright-test-generator — converts a test plan into Playwright spec files under tests/e2e/tests/<feature>/. Drives each scenario through MCP browser tools and emits TC-NN-titled tests using the authedPage fixture and the API-seed pattern.
• playwright-test-healer — runs failing specs, debugs them with snapshots / console / network introspection, and edits the spec to fix selector drift, timing, or state-leak issues.

The agents rely on the Playwright-test MCP server (mcp__playwright-test__* tools). Configure it in your Claude MCP settings; the permission allowlist lives in .claude/settings.local.json.

How to run E2E tests?

Running All Tests

With the stack already up, from tests/e2e/:

yarn test                 # headless, all projects

Running Specific Projects

yarn test:chromium        # chromium only
yarn test:firefox
yarn test:webkit

Running Specific Tests

cd tests/e2e

# Single feature dir
npx playwright test tests/alerts/ --project=chromium

# Single sub-area
npx playwright test tests/alerts/history/ --project=chromium

# Single file
npx playwright test tests/alerts/page.spec.ts --project=chromium

# Single test by title grep
npx playwright test --project=chromium -g "AL-01"

Iterative modes

yarn test:ui              # Playwright UI mode — watch + step through
yarn test:headed          # headed browser
yarn test:debug           # Playwright inspector, pause-on-breakpoint
yarn codegen              # record-and-replay locator generation
yarn report               # open the last HTML report (artifacts/html)

Staging fallback

Point SIGNOZ_E2E_BASE_URL at a remote env via .env — no local backend bring-up, no .env.local generated, Playwright hits the URL directly:

cd tests/e2e
cp .env.example .env      # fill SIGNOZ_E2E_USERNAME / PASSWORD
yarn test:staging

How to configure different options for E2E tests?

Environment variables

Variable	Description
SIGNOZ_E2E_BASE_URL	Base URL the browser targets. Written by bootstrap/setup.py for local mode; set manually for staging.
SIGNOZ_E2E_USERNAME	Admin email. Bootstrap writes admin@integration.test.
SIGNOZ_E2E_PASSWORD	Admin password. Bootstrap writes the integration-test default.
SIGNOZ_E2E_SEEDER_URL	Seeder HTTP base URL — hit by specs that need per-test telemetry.

Precedence in playwright.config.ts, lowest to highest: .env (user-provided, staging) → .env.local (bootstrap-generated, local mode) → whatever is already in process.env. The config parses both files itself and only fills in keys the environment does not already define, so exporting a variable always wins:

# runs against a locally served frontend, not whatever .env.local points at
SIGNOZ_E2E_BASE_URL=http://127.0.0.1:3301 pnpm test tests/alerts

This is deliberately not dotenv.config({ override: true }). That flag makes the file beat process.env, which silently discarded exported values — including the SIGNOZ_E2E_BASE_URL in pnpm test:staging, whenever a .env.local happened to exist.

Playwright options

The full playwright.config.ts is the source of truth. Common things to tweak:

• projects — Chromium / Firefox / WebKit are enabled by default. Disable to speed up iteration.
• retries — 2 on CI (process.env.CI), 0 locally.
• fullyParallel: true — files run in parallel by worker; within a file, use test.describe.configure({ mode: 'serial' }) if tests share list pages / mutate shared state.
• trace: 'on-first-retry', screenshot: 'only-on-failure', video: 'retain-on-failure' — default diagnostic artifacts land in artifacts/results/<test>/.

Pytest options (bootstrap side)

The same pytest flags integration tests expose work here, since E2E reuses the shared fixture graph:

• --reuse — keep containers warm between runs (required for all iteration).
• --rebuild — recreate the SigNoz container from the current sources (backend and, with --with-web, frontend) while the rest of the stack stays up. Requires --reuse.
• --teardown — tear everything down.
• --clean — prune the docker build caches, forcing the next image build to start cold.
• --with-web — build the frontend into the SigNoz container. Required for E2E; integration tests don't need it.
• --sqlstore-provider, --postgres-version, --clickhouse-version, etc. — see docs/contributing/tests/integration.md.

What should I remember?

• Always use the --reuse flag when setting up the E2E stack. --with-web adds a ~4 min frontend build on a cold start; later builds are incremental.
• Changed backend or frontend code? Re-run the setup command — it passes --rebuild, swapping the SigNoz container for one built from your current sources while the rest of the stack stays up.
• Don't teardown before setup. --reuse correctly handles partially-set-up state, so chaining teardown → setup wastes time.
• Prefer UI-driven flows. Playwright captures BE requests in the trace; a parallel fetch probe is almost always redundant. Drop to page.request.* only when the UI can't reach what you need.
• Use page.waitForResponse on UI clicks to assert BE contracts — it still exercises the UI trigger path.
• Title every test TC-NN <short description> — keeps the suite navigable and reportable.
• Split by resource, not by regression suite. One spec per feature resource; cross-resource junction concerns (cascade-delete, linked-edit) get their own file.
• Use short descriptive resource names (alerts-list-rule, labels-rule, downtime-once) — no timestamp disambiguation. Each test owns its resources and cleans up in try/finally.
• Never commit test.only — a pre-commit check or CI runs with forbidOnly: true.
• Prefer explicit waits over page.waitForTimeout(ms). await expect(locator).toBeVisible() is always better than waitForTimeout(5000).
• Unique test names won't save you from shared-tenant state. When two tests hit the same list page, either serialize (describe.configure({ mode: 'serial' })) or isolate cleanup religiously.
• Artifacts go to tests/e2e/artifacts/ — HTML report at artifacts/html, traces at artifacts/results/<test>/. All gitignored; archive the dir in CI.