Home / Services / Playwright visual regression testing

Playwright visual regression testing, on the suite you already have

You get a set of baselines that pass in your own pipeline, and a written rule saying when one of them gets updated and who decides.

Short answer

We add a visual layer to a Playwright suite you already run: which screens are worth a baseline, baselines generated in the container your CI uses, masks and tolerance policy set in one config, and a route for a failed diff to reach a person. Engineers are billed hourly, from $50 an hour, minimum one full-time engineer for one month.

Tell us which screens keep breaking What the suite audit covers

Who this is for

The engagement starts on a suite that already works.

Your design system keeps shipping regressions no functional assertion sees, and you have priced Chromatic or Percy and worked out that the licence is the smaller half of the bill. You are asking who does the other half.
You turned on toHaveScreenshot() six months ago. The snapshot folder is full of filenames ending in -darwin, a slice of the suite has never passed in CI, and --update-snapshots has become a reflex. Somebody is going to propose deleting all of it, and you want the half that works kept by an engineer who will say which half that is.

If there is no Playwright suite yet, a visual layer is a conversation about the suite first, and that is a suite built and handed over with a runbook. If the suite goes red for reasons nobody trusts, the visual layer will inherit every one of them, so the flaky work comes first. If the people who approve a UI change are designers who will never open a repository, read the product section below before you write to us.

Four people and one screen, with no design system, is too small for this.

What you get

Each of these arrives in your repository, sized to the interface it has to cover.

The visual layer, as a project of its own

The specs that hold the screenshot assertions, the baselines committed beside them, and the policy in playwright.config.ts rather than in thirty test files. It is wired as a separate project, so the layer can be run on its own, updated on its own and switched off for the week of a redesign without anybody touching the functional suite.

Baselines that hold in your pipeline

Generated inside the pinned container image your CI runs, committed, and green there before a second screen is added. A baseline belongs to the browser, the platform and the image that made it: when a Playwright upgrade moves that image, the pixels can move with it, so the regeneration command is part of the handover and the upgrade is planned around it.

A written rule for updating a baseline

Somebody has to say whether a diff is a change that was wanted, a bug the test has just caught, or a snapshot that should be masked, narrowed or taken out altogether. The rule names who says it, where they look, and what happens next in each case.

How it works

Scope
We run your suite, read the interface it drives, and come back with the list of what carries a baseline and what never should, with the reason written beside each one. A frame holding a relative timestamp, a third-party avatar or a list ordered by a real database is a red build waiting to happen, and it comes off the list. You give us a checkout of the suite, and half an hour with whoever knows which screens matter.
Baselines, where they will hold
Generated inside the same pinned image your pipeline uses, committed, and passing in CI before anything else is built on top. It is what turns a snapshot folder nobody can reproduce back into evidence. You give us the image tag your pipeline runs, or a decision to pin one if it does not have one yet.
Policy in one config
Masking and tolerance settings move into playwright.config.ts, so the policy sits in one reviewable place. A per-test override survives only where that snapshot has a named reason, because an unstable snapshot usually wants masking or deleting instead of a larger number. You give us a decision on who reviews a diff.
Handover
The rule for updating a baseline, written down and in the repository next to the runbook entry for a red build. Then a run that shows the layer passing on two separate days, so the first green is not the only evidence you have.

None of those phases carries a duration. The length depends on how many components and screens end up with a baseline, whether the application has a seedable environment that renders the same thing twice, whether the suite already runs in a container, and how much of the interface is drawing content somebody else serves.

The layer as a project of its own

The visual specs run as their own project. That is eight lines of config, and it lets the layer be updated or switched off without the functional suite noticing. The config below was run for this page on Playwright 1.62.1, on Windows 11.

import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  use: { baseURL: 'http://127.0.0.1:4173' },
  projects: [
    {
      name: 'chromium',
      testIgnore: /.*\.visual\.spec\.ts/,
      use: { ...devices['Desktop Chrome'] },
    },
    {
      name: 'visual',
      testMatch: /.*\.visual\.spec\.ts/,
      use: { ...devices['Desktop Chrome'], viewport: { width: 1280, height: 800 } },
    },
  ],
});

Playwright 1.62.1 · playwright.config.ts

testMatch and testIgnore are Playwright's documented way of splitting tests into projects by filename. The functional project skips every *.visual.spec.ts; the visual project runs only those, at a viewport pinned after the device spread so the override is the value that lands. npx playwright test --project=chromium then runs your suite with the visual layer switched off, which is the lever you want on the day a redesign merges.

The project name has a consequence, and it is cheaper to take it on day one. A baseline filename carries the browser and the platform that made it, and Playwright's visual comparisons guide adds that "If you use multiple projects in your configuration file, project name will be used instead of chromium". With the config above, the first run on this machine wrote plan-card-team-visual-win32.png: visual in the slot chromium would have taken, and win32 because that is the platform it was made on. Rename the project after two hundred baselines are committed and you have renamed two hundred files.

The platform half of that filename is the reason phase two exists. What the assertion does underneath, and what masking and the tolerance options cost you, are taken apart in baselines, tolerances and what to snapshot, which is written for the engineer doing it themselves.

Where this stops

A screenshot assertion is a narrow instrument and it is sold here as one.

It catches what moved since the last run. A layout that shipped wrong sits wrong inside the baseline, and every run after that agrees with it. Design review is a different job and this company does not sell it.
Every mask is a blind spot you chose. Masking costs less than a larger tolerance and it is legible to whoever opens the failure later, and the pixels behind the box are pixels nothing checks. Each one we add is listed in the handover rule, so the blind spot stays a decision somebody made.
The screenshot assertion belongs to the JavaScript test runner. We deliver in all four of Playwright's official bindings, and this assertion is part of the JavaScript and TypeScript runner. The assertion lists published for Python, Java and .NET do not carry it, and what they carry instead is the ARIA-snapshot assertion, which compares the accessibility tree. Say which binding your suite is in and we will tell you what we would propose.
No native mobile applications. Playwright drives browsers. A baseline can be taken at a phone-sized viewport through device emulation, which is real coverage for a responsive web application. It photographs an emulated browser, which means the viewport, the user agent string and touch events, and not a device or a store build.
Not Safari on an iPhone. If cross-browser baselines are in scope, the WebKit ones come from Playwright's WebKit build, and that is not the branded Safari your users open. Such a baseline tells you about WebKit and nothing about that phone.
A baseline says nothing about speed or safety. It compares pixels and times nothing. Load and performance work is not sold here in any framework, because a screenshot of one browser is the wrong instrument for it and k6, JMeter and Gatling are the right ones. Security and penetration work is not something this company takes on.

When a product is the better answer

Part of this market is better served by buying something, and the conditions are specific enough to state.

Chromatic, when the design system lives in Storybook and the people who approve a change are designers who will not open a repository. Its Playwright integration is a change of import, and its review app is built for that audience: snapshots indexed against git commits, approve or reject per snapshot, and accepting the changes updates the baselines.
Percy, when approval has to be a workflow across many branches and teams instead of a commit somebody merges. BrowserStack documents an approval workflow and a changes-requested workflow as first-class parts of the product.
Applitools, when the requirement is rendering checked across a large browser and viewport matrix, which is what its own platform page sells.

The licence is the smaller half of any of those. Whichever you buy, somebody still decides what gets snapshotted, gets the baselines stable and owns the review, and that is the work on this page. We build today with the comparator that ships with Playwright, inside your own repository, and we claim no integration experience with the three above.

What it costs

Nothing is priced by the screen or by the baseline. Engineers are billed hourly, from $50 an hour, depending on where the engineer sits. The minimum engagement is one full-time engineer for one month.

The size comes off your interface, your pipeline and your team. How many components and screens end up carrying a baseline. How much of the interface is drawing live or third-party content, which decides how much masking the layer needs. Whether the suite already runs in a container, or whether pinning one is part of the job. How many viewports you want covered. And whether a review route exists in your team already or has to be built from nothing.

The audit is bought on its own and priced on its own. It is optional, it is billed by the hour, and how long it takes depends on the project. The minimum of one full-time engineer for one month does not apply to it.

Read first

Baselines, tolerances and what to snapshot. Where a baseline has to be generated, what belongs inside the frame, and what each of the tolerance options loosens.
Component testing in Playwright 1.62. How a component is driven into a fixed state before anything is captured, which is most of what makes a baseline stable.
Reading a failing run. Where the expected, actual and diff images get opened, and how the trace reaches the person reviewing it.
Practices worth the argument. Why suite-wide settings belong in the config rather than in the specs.

Questions

Can our own engineers not just turn on toHaveScreenshot?

Yes, in an afternoon, and the assertion is the small part. A month later the layer survives or it does not on which screens carry a baseline and which never should, where the baselines were generated, whether the masking and tolerance policy sits in one config or in thirty test files, who opens a diff, and who regenerates the baselines when a Playwright upgrade moves the container image. The review breaks first: every other failure in a Playwright suite arrives as text a machine can route, and this one arrives as an image that needs a person with the authority to say the change was wanted.

Why do our screenshots pass locally and fail in CI?

Because the platform that made a baseline is part of its filename. One written first on macOS ends in -darwin, and the same test first run on Windows produces -win32, so a Linux CI container hunting for either finds no file to compare against: it records the snapshot as missing, writes a fresh one and goes green having checked nothing. Phase two of this engagement settles it. The baselines are made inside the pinned image your pipeline runs and committed from there, and anything generated on somebody's laptop is treated as not a baseline. Baselines, tolerances and what to snapshot covers the mechanics.

Do you use Percy, Chromatic or Applitools?

We build with the comparator that ships with Playwright: the assertion in your specs, the baselines committed beside them, and the expected, actual and diff images the runner writes when one fails, opened in the trace viewer's attachments. We claim no integration experience with any of those three products. Where one of them is the better buy, the section above says which and when, and it changes nothing about what has to be decided first: what carries a baseline, where it is generated, and who owns the review.

What is the smallest version of this engagement?

One full-time engineer for one month, which is the minimum on every engagement here bar two. A visual layer is normally scoped alongside the suite it lives in, so a team that wants two days of somebody's opinion on its snapshot folder is usually asking for the suite audit instead. The minimum of one full-time engineer for one month does not apply to it.

Do we need the audit before you start?

No, and a team that wants to know whether its suite is stable enough to carry a visual layer has a fair reason to buy one anyway. The audit is there for a team that wants to know what it has before it picks a direction. It is optional, it is billed by the hour, and how long it takes depends on the project. The minimum of one full-time engineer for one month does not apply to it.

Which screens would you want a baseline for?

Send that list, including the ones nobody snapshots today and somebody wishes they did, and say what your pipeline runs on: the image tag, or the runner if there is no container in it yet. The list and the image tag settle the first two phases between them: what carries a baseline, and where the baselines have to be generated to hold. We do not need an audit to start.

hello@firm86.com What the audit covers