Key principles
Merrykat is visual regression and accessibility testing for Storybook. Install it as a GitHub app and add it to CI and you’re all set.
Less focus on screenshots
A run is a pair of static Storybook builds: your pull request’s head and its merge base. These builds render side by side in the same browser, at the same moment. The pixels are (mostly) thrown away afterwards; only the handful of images shown in the pull request comment survive.
Three properties follow which all factored into the design of Merrykat:
- No renderer drift. Both sides render in one container, one browser, one hardware instance, back to back. That is what makes an exact hash comparison viable as the primary gate, instead of a tolerance threshold tuned to absorb the difference between two renders.
- No baseline approval workflow. There is no golden image to record, re-record, or forget to re-record. The baseline is whatever the pull request’s merge base built, and when the pull request merges, the new default-branch build becomes the baseline for everything after it.
- Runs are reproducible. The inputs are two immutable tarballs, so a run can be replayed against a newer diff engine or a newer prompt and the difference attributed.
Diffs are information
A visual difference is not a failure: it is usually the point of the pull request. The check run goes neutral on differences and reports them; it fails on genuine breakage: a story that throws, a story that renders nothing, or a high-severity likely bug. You can control when builds fail with the failOn configuration option, and the configuration page covers it.
“Broken” means newly broken
A story that already threw on your default branch is reported and never blamed on your pull request. The same rule holds for blank renders and for accessibility violations: present on both sides means pre-existing, and pre-existing never fails a check.
The check outcome never depends on the model
Whether a story errored or came back blank is decided by the render engine, and those drive failOn directly. The model’s job is to choose what is worth looking at and to explain what it sees; its severity judgement on a borderline story can vary between runs, and a merge decision must not. A high-severity likely bug is an additional reason to fail, never the only one.
With the AI layer switched off entirely, a run still produces the full diff report and it falls back to one representative story per change class instead of a chosen shortlist.
Stories render with no network access
Rendering a Storybook means executing your bundle, so it runs with outbound network access blocked outright. That is a security control and a determinism one at once: a CDN that is slow today is not a flake source tomorrow. A Storybook that loads webfonts remotely renders with fallbacks, identically on both sides, so the comparison stays valid, and the blocked hosts are named in the comment so the difference is never a mystery.
The GitHub App has only the bare minimum permissions
The App holds no permission to read repository contents at all. Baseline resolution and the pull request’s own diff are computed by the CLI. It is opt-out: set ai.codeContext: false, and the review still runs without it.