Compare commits

...

2 Commits

Author SHA1 Message Date
Abhi Kumar
11ed0b0c62 docs(e2e): assert one run instead of re-running on cache hits 2026-10-30 16:39:17 +05:30
Abhi Kumar
9f04647e5b docs(e2e): add timing traps to playwright agent docs 2026-10-30 16:33:21 +05:30
2 changed files with 13 additions and 1 deletions

View File

@@ -150,6 +150,18 @@ Note: no cleanup code (the `dashboards` driver deletes its seed), no raw locator
- **Monaco editor swallows Escape.** Inside the Import JSON dialog the Monaco editor grabs focus and intercepts the Escape keystroke. Click the modal title first to blur Monaco; `dashboardList.dismissWithEscape()` does this.
- **Empty zero-state hides controls.** With no dashboards in the workspace, the list controls and `new-dashboard-cta` are absent. Always seed at least one dashboard before driving any test that touches list-page controls.
# Timing traps
Each of these passes alone and flakes under parallel CI load. Handle them inside the driver action.
- **Wait for the request the action started.** A `waitForResponse` registered after the click, or matching any in-flight request, can resolve on an earlier one. Register `page.waitForRequest(...)` before the action, then await its `response()`.
- **Let React commit before a shortcut.** A keyboard shortcut (e.g. Cmd+Enter) pressed right after typing can read the previous state. Wait two animation frames (`requestAnimationFrame` twice in `page.evaluate`) before pressing it.
- **Typing before the first load is lost.** Pages that reset inputs when their initial data lands wipe early keystrokes. The `open` action waits for the page's first data request.
- **A cache hit fires nothing.** When a page already fetched on an earlier edit, the next Run may be served from the React Query cache. After one Run, assert the page shows the expected result and the last request sent carries the edit. Don't wait for a fresh request that may not come, and don't click Run repeatedly: that hides a first click sending stale values.
- **Open autocomplete explicitly.** Suggestions that appear on focus or typing arrive late under load. Trigger them (e.g. Ctrl+Space) and wait for the option.
- **antd dropdowns render in a portal.** A Select's options are not inside the Select. Scope to that Select's own open dropdown, and assert the chosen value on `.ant-select-selection-item`.
- **Split long scenarios.** A test chaining many reloads or edits nears the timeout under load. Split it.
# Quality bar
- Every test runs end-to-end against a fresh stack. If you can't run it green from a fresh `test_setup`, it's not done.

View File

@@ -28,7 +28,7 @@ Don't try to start the stack yourself — it can take ~4 minutes on a cold build
- `browser_generate_locator` to suggest a stable locator if the failing one drifted.
4. **Root-cause.** Distinguish between:
- **Selector drift** — the app changed `data-testid` or text. Fix the locator where it lives: in the driver under `tests/e2e/drivers/` when the spec uses one, so every spec sharing it heals at once. Prefer `data-testid` (grep `frontend/src/<feature-dir>/` for the new one).
- **Timing** — the test races a load. Replace `waitForTimeout` with `await expect(locator).toBe…()` or `page.waitForResponse(...)` on the triggering action — inside the driver action when there is one.
- **Timing** — the test races a load. Replace `waitForTimeout` with `await expect(locator).toBe…()` or `page.waitForResponse(...)` on the triggering action — inside the driver action when there is one. A test green alone but red with `CI=1` is usually one of the generator's [timing traps](playwright-test-generator.md#timing-traps).
- **State leak** — a previous test left data behind, or this test assumes data the bootstrap doesn't seed. Seed through the driver (its `dispose` cleans up) or via API with `try / finally`. The bootstrap creates a fresh stack with **zero** dashboards / alerts and only the golden telemetry; parallel per-test seeds share the tenant, so an exact-count assertion on telemetry is a test bug — assert structure instead.
- **Genuine app bug** — the app is broken, not the test. Mark the test with `test.fixme(...)` and add a one-line `// known: <description>` comment. Don't silently change the assertion to make it pass.
5. **Fix.** Edit the spec or its driver. Preserve TC-NN titles, the driver / `authedPage` fixtures, cleanup, and serial mode if present. A driver change must keep every other spec importing it green — run them too. If you renumber, retitle, or `test.fixme(...)` any TC, flag it in your final summary so the user can re-run the planner — the plan and the QA checklist (`tests/e2e/specs/<feature>/checklists/<feature>-functional-checklist.md`) re-derive from the current `.spec.ts` and will otherwise drift.