Playwright CI/CD integration that stops the build being a checkbox

The decisions, in one config block

There is no workflow file on this page. Playwright's own CI walkthrough is a complete GitHub Actions setup that works on the first push and costs nothing, and arguing with it would be arguing with something you have already used. The sample below is the other half: the part of playwright.config.ts that behaves differently in a pipeline than on a laptop. Every line in it is a decision, and none of them is in the YAML.

import { defineConfig } from '@playwright/test';

export default defineConfig({
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  failOnFlakyTests: !!process.env.CI,
  workers: process.env.CI ? 1 : undefined,
  reporter: process.env.CI
    ? [['html', { open: 'never' }], ['junit', { outputFile: 'results.xml' }]]
    : [['list']],
  use: {
    trace: 'on-first-retry',
  },
});

A test.only left in a file passes on a laptop and silently skips the rest of the suite, so the build is green and the build did not run. forbidOnly, keyed on the CI variable, fails that run instead.

retries is the declared version of the re-run habit. A retry in the config is counted and reported; a person clicking re-run is not. failOnFlakyTests is the decision underneath it: with that on, a test that only went green on a retry fails the run instead of being filed as flaky and forgotten. Whether that counts as green is a policy question, and it is one of the ones this engagement makes somebody answer.

workers and reporter are both about who is on the other end. Playwright's CI guidance is a single worker, on the reasoning that a test with the whole machine to itself is more reproducible, and that wider parallelism should be bought by sharding across jobs instead. Whether it holds on your runners is measured in phase two. The reporter already differs between a laptop and a pipeline by default, and the decision is which output your team reads: an HTML report a person opens, a machine-readable file the pipeline turns into annotations on the pull request, or both.

trace set to on-first-retry keeps evidence only for the runs that needed it. It is the line that decides whether the person reading a red build at three in the morning has the failing step in front of them or a log to guess from.

What it costs

Engineers are billed hourly, from $50 an hour, depending on where the engineer sits. The minimum engagement is one full-time engineer for one month.

The estimate here comes from how many pipelines run any part of it, how many environments it has to answer for, how many browser projects are in scope, whether it passes anywhere except the laptop it was written on, and whether there is an existing pipeline to work inside or an empty repository. A green suite in one pipeline is a different engagement from the same suite in four, and the second one is priced by the number of places a decision has to be repeated.

There is a second cost and it is yours, not ours: the CI bill. Sharding cuts wall-clock time and raises billed minutes, because the variable part of a run divides across jobs and the fixed part multiplies. Every shard is a job that checks out again, installs again, and downloads a browser or pulls an image again. So the question we answer is not how many shards you can run but where the next one stops buying anything, and concurrency caps decide part of it for you, because shards above the cap queue and queueing is still time. The arithmetic behind that cut-off is in our article on parallel runs and sharding.

One thing that does not pay for itself: caching browser binaries between runs. Playwright's CI documentation advises against it, because restoring the cache takes about as long as downloading the binaries does, and on Linux the operating system dependencies that go with them are not cacheable at all. If cutting that fixed cost is where you were going to start, start somewhere else. It is a decision we make once, in your pipeline, and then stop spending engineering time on.