docs(skills): a scoped noise run marks every story it omits as flaky

.claude/skills/sbshot-visual-diff/SKILL.md

@@ -0,0 +1,164 @@
+---
+name: sbshot-visual-diff
+description: Shoot Storybook stories with sbshot at two git refs and produce a diff a reviewer can read. Use when asked for a visual diff, before/after screenshots, a screenshot comparison for a PR, or to prove a change did not alter screens it should not have.
+---
+
+# sbshot visual diffs
+
+`npx -y github:SigNoz/sbshot` shoots Storybook stories and pixel-diffs two runs.
+Use it. Do not hand-roll a Playwright screenshot loop or a canvas pixel-compare:
+sbshot already handles story selection, themes, settling, retries, diffing,
+tagging, a viewer and a static export, and its runs are the format the rest of
+the team can open.
+
+sbshot is not installed by any package. It is run through `npx`, so nothing it
+knows travels into this repo on its own. This file is where that knowledge
+lives.
+
+## The shape of a run
+
+```
+capture <storybook> --name-run <ref>     # once per git ref
+diff <baseline> <after>                  # pairs them by relative path
+tag <diff> <tag> <story-id>...           # what the reviewer should look at
+export <diff> --out <dir>                # a static site, hostable
+```
+
+`<storybook>` is a running dev server (`http://localhost:6006`), a build
+directory holding `index.json`, or a project directory with `.storybook/`. The
+frontend has **no `build-storybook` script**, only `storybook dev -p 6006`, so
+point `capture` at a dev server.
+
+## Cover the change with stories first
+
+**A diff can only show what a story renders.** A screen, state or variant with no
+story is invisible to sbshot: it produces no pair, so it reads as "nothing
+changed" rather than "never looked at". This is the failure mode that makes a
+clean-looking diff worthless.
+
+Before capturing, list the screens and states the change touches and check each
+has a story. For anything missing, write it first — the `signoz-page-story`
+skill covers how. New states reached only by an interaction need a `play`
+function, or the capture shoots the page at rest and the state never appears.
+
+Then confirm it worked: the two runs' story counts should differ by exactly the
+stories the change adds. A new screen that produced no new story id is a screen
+nobody is reviewing. Say the count in the summary, so a reader can tell coverage
+from silence.
+
+## Rules that are not optional
+
+**One freshly started Storybook per ref.** Never start one server and switch git
+branches under it. Vite's HMR reloads the page, but some modules do not
+re-initialise, so the shot differs silently from a clean boot and the difference
+looks like a real change. Settle time does not fix this and is not the variable.
+Use one worktree and one port per ref, and stop each server before the next.
+
+**Capture both themes: `--theme dark,light`.** Without it sbshot shoots the
+story's own default and everything lands in a single `default/` folder, so every
+regression in the other theme is invisible. Themes double the shot count, so
+scope with `--stories` or `--title`, never by dropping a theme.
+
+**Baseline is the merge-base, not `origin/main`.** `git merge-base origin/main
+<branch>`. Against a moved main, every commit main gained since the branch's base
+shows up reversed, as though the branch reverted it.
+
+**The percentage sorts the list, it does not tell you what matters.** A dialog
+that dims the page scores high for a one-line change, and a whole new form
+section can score low. Open the shots and say what changed on each screen. The
+biggest number is usually the least interesting one.
+
+**If you cannot explain a pair, assume the capture is at fault, not the code.**
+Open it, crop both sides and compare them before calling it noise. A pixel count
+and a good guess are not evidence.
+
+## Noise, and the floor that proves it
+
+The same story does not always render identically twice. Pairs often move for
+reasons that have nothing to do with the change:
+
+- a focus ring left on whatever a `play` function clicked last
+- a scrollbar appearing, or its thumb resting a few pixels off
+- a toast, tooltip or dialog caught mid-fade
+- a text caret blinking, or a selection highlight
+- an animation or spinner at a different frame
+- relative timestamps ("2 days ago") crossing a boundary
+- hover or active styling from where the pointer happened to rest
+- a late-loading font or image shifting a line of text
+
+Do not argue about these one at a time. **Measure them**: shoot the same commit
+twice and diff those two runs against each other. Anything that moves there moved
+for no reason, so it is noise.
+
+```
+capture <storybook> --name-run noise-a
+capture <storybook> --name-run noise-b     # same tree, its own fresh server
+diff noise-a noise-b --name-diff noise
+diff <baseline> <after> --noise noise      # marks the movers flaky
+```
+
+**The noise run has to cover the same stories as the real diff.** Do not scope it
+down to the few pairs that moved, however tempting that is. `--noise` marks a
+pair flaky when the noise diff does not show it as unchanged, so every story the
+noise run leaves out is marked flaky too, including the new screens the reviewer
+is there to look at. Shooting two narrow runs and diffing those against each
+other has the same problem. Either shoot the full set twice, or skip `--noise`
+and explain the few movers by hand.
+
+That is the trade: a second full capture costs about as long as the first, so on
+a small diff it is often quicker to crop the two or three movers and say what
+each one is. Use `--noise` when the diff is large, or when wiring `--fail-over`
+into CI, where something has to decide without a person looking.
+
+If a pair is flaky every run and the state it captures matters, fix the story
+rather than living with it: blur what the `play` clicked, freeze the clock, or
+assert the state instead of shooting it.
+
+## Making it readable for a reviewer
+
+A reviewer cannot open `localhost`, so `sbshot ui` is for you, not for them.
+
+1. `tag` the pairs that matter, with `--note` saying why. The viewer filters by
+   tag, so the reviewer lands on the real changes instead of scrolling past
+   noise that scored higher.
+2. `export <diff> --out <dir>`. It writes a static site carrying only the pairs
+   that are not identical, converted to WebP, so a large diff stays small.
+3. Host it. A Claude artifact works: publish the export's files and give the
+   reviewer the link. `index.html` is reserved for the artifact's own page, so
+   publish sbshot's entry as `viewer.html` and link to
+   `viewer.html#/job/diff/<diff>`.
+4. Put the link in the PR body under **Screenshots / Screen Recordings**.
+
+Say the numbers plainly: how many screens changed, how many are new, how many are
+pixel-identical, how many are flaky. The count of unchanged screens is often the
+most useful claim in the diff, because it is the one a reviewer cannot check by
+eye.
+
+## Operational traps
+
+- A full capture is hundreds of shots. Scope with `--title "Pages/Alerts"` or
+  `--stories "a,b,c"` (substring match on id or Title/Name, repeatable).
+- A fresh worktree cannot build Storybook: `i18n-translations-hash.json` is
+  gitignored and generated. Run `node ./i18-generate-hash.cjs` in
+  `<worktree>/frontend`, and symlink `node_modules` from the main checkout.
+- `pnpm test:storybook` starts its own server on 6006 and hangs silently if
+  something already holds that port. Free it first.
+- ImageMagick on macOS may have no default font, so `--mode green-parallel` fails
+  with ``unable to read font `' ``. Map Helvetica to a real file in
+  `~/.config/ImageMagick/type.xml`.
+- `sbshot link <dir> <name>` adds a run from elsewhere into the workspace, but it
+  symlinks. Keep the source directory alive until you have exported.
+- The export directory is build output. Delete it when done, or gitignore it.
+- Running the whole Storybook test suite locally fails broadly with
+  `page.goto: Timeout` from resource starvation. Scope it; CI is the real signal.
+
+## Done means
+
+- [ ] every screen and state the change touches has a story, written before capturing
+- [ ] one capture per ref, each from its own freshly started Storybook
+- [ ] `--theme dark,light` on every capture
+- [ ] baseline is the merge-base
+- [ ] noise either measured with a full second capture, or each mover explained by hand
+- [ ] every non-flaky pair opened and explained, not inferred from its size
+- [ ] the pairs that matter tagged, with a note
+- [ ] exported and hosted, and the link in the PR body

.claude/skills/signoz-page-story/SKILL.md

@@ -151,9 +151,42 @@ that only exists on a trial needs the plan too:
 `args: { plan: 'on-trial', revoked: ['create:subscription'] }`. A permission
 whose denial changes nothing on screen gets no story: say so in the PR.
 
-Verify these by their disabled states, not their text. The page reads the same
-either way, so a story that is wrong looks right: read `disabled` off the buttons
-the permission gates, and check the denial callout is there or gone.
+Check these by what is turned off, not by what the page says. The page reads the
+same either way, so a story that is wrong still looks right. Check that the
+control the permission guards is off, and that the denial message is there or
+gone.
+
+**Do not use `toBeEnabled` or `toBeDisabled`.** They only look at the `disabled`
+attribute. `@signozhq/ui` turns a button off with `aria-disabled` instead, so
+these matchers pass on a button nobody can click, and the test goes green while
+clicking straight through a denial. Check the attribute the component really
+sets:
+
+```ts
+expect(save).toHaveAttribute('aria-disabled', 'true');
+```
+
+**Some controls are off with no attribute at all.** The browser turns off
+everything inside a disabled `<fieldset>` without marking it, and a switch is a
+`<button>` underneath. For those, an attribute check proves nothing, and so does
+looking for a parent `fieldset[disabled]`. Click it and check nothing happened:
+
+```ts
+const before = toggle.getAttribute('aria-checked');
+await userEvent.click(toggle);
+await expect(toggle.getAttribute('aria-checked')).toBe(before);
+```
+
+**Check that a new assertion can fail.** Flip it once and run it. If that story
+goes red and nothing else does, the assertion is real; put it back. A test that
+cannot fail is worse than no test, because it looks like coverage.
+
+**Delete a story once its difference disappears.** Say a story checks that a
+button reads "View" rather than "Edit" when a permission is missing. If the label
+later becomes the same either way, that story passes no matter which permissions
+are held, including all of them, and it is testing nothing. When a behaviour
+stops depending on a permission, delete the story or rewrite it to lock in the
+new behaviour, and say which one you did in the PR.
 
 ## Rules
 

.claude/skills/signoz-page-story/references/verify.md

@@ -63,6 +63,10 @@ not catch an unstyled page. Take them with
 [sbshot](https://github.com/SigNoz/sbshot), e.g.
 `sbshot capture http://localhost:6006 --title <story title> --theme dark,light`.
 
+That is one run, for checking the story you just wrote. Comparing two branches is
+a different job: see the `sbshot-visual-diff` skill, which covers the baseline to
+pick, measuring the noise and exporting something a reviewer can open.
+
 ## Gates
 
 - **Console silent.** `[storybook] no msw handler`, a 501 from the catch-all, an