Home / Blog / AI test generation

AI test generation with Playwright: what 1.62 ships

Updated 1 September 2026 · Playwright 1.62

All four are Microsoft's and all four now install with @playwright/test. None of the documentation says what you are holding once one of them has finished.

Everything below ran on Playwright 1.62.1, which is what an unpinned npm install returned here on 1 September 2026, on Windows against Chromium 151. This corner of the framework moves faster than the rest of it, so if you are reading months later, check the release notes before you trust a command name. Nobody here has driven the three agents through a full loop with a coding agent: what they do inside it comes from Microsoft's documentation and from the definition files the tool wrote on disk.

What does Playwright 1.62 ship for AI test generation?

Only the first has been in the box for years. The three test agents arrived in 1.56, and the last two rows are what 1.62 changed.

1.62 changed the bundling, and the release notes carry it as one bullet under New APIs → Command line & MCP: "Playwright now bundles the Playwright MCP server and playwright-cli, runnable via npx playwright mcp and npx playwright cli." Anyone who has these filed as third-party bolt-ons can stop. The bullet is false about 1.61, so keep the version attached to it.

All four drive browsers and nothing else, so none of them reaches a native iOS or Android application, and none of them is a load or a security tool. Our explainer on what Playwright can and cannot drive has the full boundary.

You have almost certainly run the first row already. The other three are the ones that would cost you an afternoon to try.

Is playwright codegen AI test generation?

No. It opens two windows, a browser and the Playwright Inspector; you click through a flow; the Inspector writes the script as you go and you copy it out. The same actions produce the same script every time.

npx playwright codegen https://demo.playwright.dev/todomvc

Playwright 1.62.1 · shell

That is the first sample here, so: every command and every spec on this page was written against Playwright 1.62 and run on 1.62.1. VS Code carries the same recorder in its Testing sidebar, and docs/codegen documents it under Record a New Test and Record at Cursor — the cursor in your open file, not the editor of that name, which turns up later in the MCP client list.

docs/codegen uses the words AI, LLM and model nowhere in its body, and that is a finding rather than an inference: the page was pulled whole with curl on 1 September 2026, 66,603 bytes, ending on the page.pause() sample under Record using custom setup, then searched. MCP is on it exactly once, in the navigation bar, so anyone citing that page as evidence that the recorder is MCP-aware is citing the furniture.

What do the planner, generator and healer do?

Three agent definitions and their tooling, written into your repository by one command. Here it is on 1.62.1, in an empty project with @playwright/test installed and nothing else.

$ npx playwright init-agents --loop=vscode
 🎭 Using project "" as a primary project
 📝 specs\README.md - directory for test plans
 🌱 seed.spec.ts - default environment seed file
 🤖 .github\agents\playwright-test-generator.agent.md - agent definition
 🤖 .github\agents\playwright-test-healer.agent.md - agent definition
 🤖 .github\agents\playwright-test-planner.agent.md - agent definition
 🔧 .vscode\mcp.json - mcp configuration
 🔧 .github\workflows\copilot-setup-steps.yml - GitHub Copilot setup steps

Playwright 1.62.1 · shell, on Windows · trimmed after the file list

After that list it prints an MCP configuration block to paste into GitHub > Settings > Copilot > Coding agent, and then it finishes. Seven files, no prompts.

The documentation shows four tabs for --loop: VS Code, Claude Code, Codex and OpenCode. The binary takes six. npx playwright init-agents --help on 1.62.1 lists claude, codex, copilot, opencode, vscode and vscode-legacy. The docs also say to regenerate the definitions on every Playwright upgrade, which makes them a committed build artifact.

What each agent is for, from docs/test-agents:

• Planner. Takes a request, a seed test that sets up the environment, and optionally a requirements document, then writes a Markdown plan into specs/. It runs the seed test to do the initialisation and uses it as the example the generated tests are modelled on.
• Generator. Turns that plan into files under tests/, verifying selectors and assertions live as it performs the scenarios.
• Healer. Runs a failing test and tries to repair it, which is the next section.

The seed test is the part a reader skims: the planner cannot plan until something has put the application into a state worth exploring. Here is the seed file init-agents wrote, in full.

import { test, expect } from '@playwright/test';

test.describe('Test group', () => {
  test('seed', async ({ page }) => {
    // generate code here.
  });
});

Playwright 1.62.1 · TypeScript · seed.spec.ts, written by init-agents

The comment is where your login, your fixtures, your global setup and your test data go. Somebody writes that by hand. Generation starts from a hand-written test.

The generator's documented output is two bullets: "A test suite under tests/", and "Generated tests may include initial errors that can be healed automatically by the healer agent". The vendor is saying its own output may be wrong, and the pipeline is drawn with a repair stage in it.

Playwright MCP or playwright-cli: which do you want?

MCP is a server. A language model calls tools on it with structured parameters and gets back an accessibility-tree snapshot: text with a reference on each interactive element, no pixels and no coordinate guessing. You set it up with a JSON block in an MCP client, and the browser starts headed.

playwright-cli is a command surface. The agent runs shell commands, reads short output and picks up capabilities as installable skills. It starts headless and is aimed at coding agents working inside a large codebase.

Both vendor pages carry the same comparison: what each is best for, how the agent talks to it, what it costs in context window, headed or headless, and the setup. It is a fair table, and it is Microsoft comparing two of its own products, so it has nothing to say about the recorder further up this page. The setup guides are docs/getting-started-mcp and docs/getting-started-cli.

Either matters to a test suite for one reason. They let a coding agent drive a real browser, which is what makes the generator's live verification possible at all. A third server sits behind the agents themselves: the .vscode/mcp.json that init-agents wrote points at npx playwright run-test-mcp-server.

What does a generated test get right, and what can it not know?

It does the transcription well.

• The selectors. A tool watching a live browser resolves elements better than a person guessing from markup, and the generator confirms each one by performing the step.
• The sequence. A tool transcribing a live session has the actions in the order they happened, including the steps a person writing from memory leaves out.
• The boilerplate. Imports, the describe block, navigation, the shape of a spec file.
• The blank page. For a team with no suite at all, the distance from zero to something that runs is where this tooling pays fastest.

Here is the shape of a first draft, written by hand for this page: a flow through the demo application Playwright's own docs use, asserting what was observable while it ran.

import { test, expect } from '@playwright/test';

test('Add Valid Todo', async ({ page }) => {
  // 1. Open the app
  await page.goto('https://demo.playwright.dev/todomvc');

  // 2. Type "Buy groceries" into the input and press Enter
  const input = page.getByRole('textbox', { name: 'What needs to be done?' });
  await input.fill('Buy groceries');
  await input.press('Enter');

  // Expected results
  await expect(page.getByText('Buy groceries')).toBeVisible();
  await expect(page.getByText('1 item left')).toBeVisible();
});

Playwright 1.62.1 · TypeScript · tests/draft.spec.ts · passes

It runs, it is green, and it is an accurate record of what happened. Now the same flow after somebody who knows the product has been through it.

import { test, expect } from '@playwright/test';

test('a todo is still there after a reload', async ({ page }) => {
  await page.goto('https://demo.playwright.dev/todomvc');
  const input = page.getByRole('textbox', { name: 'What needs to be done?' });
  await input.fill('Buy groceries');
  await input.press('Enter');

  await page.reload();

  await expect(page.getByTestId('todo-title')).toHaveText(['Buy groceries']);
  await expect(page.getByTestId('todo-count')).toHaveText('1 item left');
  await expect(page.getByRole('checkbox', { name: 'Toggle Todo' })).not.toBeChecked();
});

Playwright 1.62.1 · TypeScript · tests/owned.spec.ts · passes

Three things changed. The assertions moved off this text is on the screen and onto the list contents and the counter, so a second todo appearing from nowhere fails the test. A reload went in, because this application promises that your list survives one, and no recorded session presses refresh. And the title now names the promise being guarded.

The difference is measurable. Disabling the app's storage writes with a one-line init script, Storage.prototype.setItem = () => {}, stands in for a persistence regression; under it the second spec fails after the reload, at getByTestId('todo-title'), with locator resolved to 0 elements, and the first spec passes. The draft never reloads, so that build has nothing in it to break.

The rest of a test is judgement:

• Which assertion matters. A generated test asserts what was observable. It does not know that the number in the corner is the invoice total and the one beside it a cache-warmed estimate, or that only one of the two being wrong is an incident.
• Which flows earn money. Checkout and password reset are different in kind, and nothing in a recorded session ranks them. The ranking is the test strategy.
• What must never be mocked. Whether the payment call may be stubbed is a business decision with somebody's name against it, and nothing in a recorded session knows whose name.
• What the product is supposed to do. It records what the application did, so on a screen with a bug in it the bug becomes the expected value. That is the failure that survives review, because the test is green.

What can a self-healing repair hide?

When a test fails, docs/test-agents says the healer agent:

• "Replays the failing steps"
• "Inspects the current UI to locate equivalent elements or flows"
• "Suggests a patch (e.g., locator update, wait adjustment, data fix)"
• "Re-runs the test until it passes or until guardrails stop the loop"

Its output is documented as "A passing test, or a skipped test if the healer believes that functionality is broken."

Now the argument, which is ours. A test suite is an alarm, and a loop that runs until the alarm stops is being scored on the wrong signal. The four patch kinds come apart under that:

• A locator update is usually right. A button moved, a class changed, nothing a user would notice.
• A wait adjustment is where it turns. A step that now needs longer is either a race the test was correctly catching or a page that got slower for everybody, and both look identical from inside the repair loop.
• A data fix is the one to think hardest about. If the expected value changed and the repair updates the expected value, the suite goes green and the assertion has been rewritten to match the product.
• Skipping is the one nobody notices. A suite that quietly skips has a green tick worth less than yesterday's, and nobody reads the skip count.

None of that needs guesswork about the agent's intentions, because the instructions ship as a file you can read. On 1.62.1, .github/agents/playwright-test-healer.agent.md lists "Fixing assertions and expected values" among the remediations, tells the agent "You will continue this process until the test runs successfully without any failures or errors", and tells it to mark a test test.fixme() when the error persists and it is confident the test is correct.

So a healed test is a pull request, not a result. Every repair is a diff a person reads, and the question to ask of each one is what changed in the product to make it necessary — the same question a red build that comes back green on a retry should raise, and that article separates the four causes and what to do about each.

What does a generated suite cost to maintain?

The pitch for generation is a cost argument about writing tests. The larger bill is that a suite has to be read: every time it fails, every time the application changes, every time somebody new joins. Generation does nothing to reduce that. Nobody here has measured this, so take what follows as reasoning to check against your own suite.

Nobody on the team wrote it. The best predictor of how fast a failing test gets diagnosed is whether someone in the room remembers writing it. A generated suite starts at zero on that, for every spec, on day one.

Volume is the easy axis. Generating the two-hundredth test costs almost nothing and maintaining it costs what maintaining the first did, so the tooling pushes on exactly the number a maintenance bill is proportional to.

Uniform is not the same as conventional. Generated specs are consistent with the generator's idea of structure, which may not be your fixtures, your page objects or your naming. A suite that half-follows two conventions is harder to read than one that follows either.

The plans are a second artifact. The planner's Markdown files under specs/ are real files in the repository, and nothing keeps them in step with the tests once a person has edited either side.

Where does generation belong?

Use it where the work is transcription: the first pass over a flow, the repository with no suite in it, the selectors for a screen nobody has automated. Keep the judgement with people — which flows, which assertions, what may be mocked, and every healed diff. Read what it produces before it merges, at the standard you would hold a new hire's first pull request to.

That standard is easier to hold when somebody has written the answers down, which is what the runbook a suite is handed over against is for: how to add a test, which locator to reach for, and how to read a failure. Whether a spec was typed or generated stops mattering once the team can read it.

Questions