Learn

Baselines and approval

Where baselines live, who approves a change, how to keep history, and why the review step is the part of the workflow most teams get wrong.

A baseline is a decision frozen into a PNG: "this is what it should look like". The workflow around baselines decides whether visual tests stay useful or turn into a chore.

Where baselines live

In the repository, next to the tests. Baselines version with the code: a branch that changes the header carries the header's new baselines, a revert reverts them, and git blame says who approved what. Every local tool (Playwright, Vitest, jest-image-snapshot, BackstopJS, Roborazzi, pixsame) works this way. The costs are repository size, which Git LFS fixes if it becomes a problem, and the need for every contributor to produce identical pixels, which a pinned renderer fixes.

In a service, keyed by branch and commit. Hosted review tools (Percy, Chromatic, Argos, Applitools, Happo) and self-hosted reg-suit store baselines on their side, compare a build against the baseline of the branch it was forked from, and keep the history. Nothing lands in the repository, developers do not need matching environments, and the review UI comes with it. The costs are a dependency on the service for every run, pricing per screenshot, and baselines that are not reviewable as part of the code diff.

Both work. Repository baselines suit teams who want the whole state in git; service baselines suit teams with many contributors on many machines. The comparison lists which tools do which.

Who approves

The reviewer of the pull request, not the author. A visual change approved by the person who made it is a visual change nobody looked at. The ideal flow:

The PR runs the tests. Failed comparisons are reported where the review happens: a check on the PR, a comment with the images, or a link to the review app.
The reviewer looks at baseline, actual and diff. The slider between baseline and actual is the fastest way to see what moved.
Each change is either a bug (the PR gets fixed) or intended (the reviewer approves it, and the actual image becomes the baseline).
The approval is recorded: a commit that updates the baseline files, or a status in the service.

Tools that only fail the build and leave the images in a CI artifact push step 2 onto whoever digs the artifact out, which is nobody. The reporting step is where the manifest comes in: a file that lists which comparisons need a human and where their images are, so a PR bot or review page can be built once and used with any tool.

Approving from a pull request

With repository baselines, "approve" means copying the actual image over the baseline and committing it to the PR branch. This can be a local command (npx playwright test --update-snapshots, Roborazzi's recordRoborazzi*, BackstopJS's approve), a CI job triggered by a comment or label that commits the files back, or a GitHub App that does it from a button. The manifest's images.actual.path and images.baseline.path are exactly the two paths such a job needs; summary.needsHuman says whether there is anything to do.

With service baselines, approval is a click in the service's UI, and the service marks the build as passed.

Partial approval and the "approve all" trap

A PR that changes a shared component can fail fifty screenshots. If the only button is "approve all", the fifty approvals happen without fifty looks. Prefer tools and flows that allow approving per screenshot, group the failures by cause (the manifest's keyHash and selectEntries exist for this), and make the summary show the biggest diffs first.

Baseline hygiene

Delete baselines of deleted tests. Orphaned PNGs accumulate silently. Some tools warn about unused baselines; a script that lists baselines no test references is five lines.
Name baselines after what they show, not when they were taken. checkout-summary-mobile.png survives refactors; test-3-0.png does not.
Keep one baseline per thing you actually test. One per browser and OS if you really run several; otherwise pin the renderer and keep one.
Review baseline changes like code changes. A PR that touches 40 PNGs deserves a reviewer who opens them. Services show them side by side; in GitHub, the rich diff for images does.