What is a browser test framework?

What is a browser test framework?

A browser test framework is an open-source library that lets test code start a real browser, act on web pages, and check the results. It is also called a browser testing framework. Playwright, Selenium, and Cypress are three frameworks of this kind.

The three differ in what they include. Playwright and Cypress each ship a test runner, which finds the test files, runs them, and reports the results. Playwright's runner is for JavaScript and TypeScript, and its Python, Java, and .NET versions plug into an existing runner, e.g. pytest. Selenium's WebDriver library controls the browser only, so a team pairs it with a separate runner, e.g. JUnit.

Most browser tests are end-to-end tests of a web app, and each one usually follows one user journey, e.g. checkout. A browser test framework is one part of test automation for the web. A command-line application needs no browser, because its tests run the program and read its text output.

How does a browser test framework work?

A browser test run follows the same steps in each framework:

1. The test runner finds the test files, and the framework starts a browser, often a headless browser with no window.
2. A test sends a command to the browser, e.g. "fill in the delivery address."
3. The framework finds the target element with a locator, a query that matches elements, e.g. by their label.
4. The framework or the test waits until the element is ready, and then the browser performs the action.
5. The test's assertions compare what the page shows with the expected values.
6. The runner reports each pass and failure, and it can save a screenshot or a trace of each failed test.

The frameworks differ in how a command reaches the browser. Playwright controls Chromium over the Chrome DevTools Protocol, and it uses its own patched builds of Firefox and WebKit. Selenium sends commands in the W3C WebDriver standard to a driver, e.g. ChromeDriver, and most drivers come from the browser vendors. Cypress runs the test code inside the browser, next to the app, with a Node.js process beside it for work outside the page.

What is an example of a browser test?

Here is an illustrative example. Acme Co. sells furniture online. A developer asks a coding agent to "Let customers edit their delivery address during checkout."

In this version of the example, Acme's browser tests use Selenium's JavaScript library, with Jest as the test runner. A Jest setup file starts Chrome through ChromeDriver and stores the session as driver. The agent changes the checkout page and writes this test:

import { By, until } from "selenium-webdriver";

test("saving the address shows a confirmation", async () => {
  await driver.get("https://shop.example.com/checkout");
  await driver.findElement(By.id("address")).sendKeys("12 Elm Street");
  await driver.findElement(By.id("save-address")).click();
  await driver.sleep(200);
  const saved = await driver.findElement(By.id("address-saved"));
  expect(await saved.isDisplayed()).toBe(true);
});

The test passes when the agent runs it. On the build server, the same test passes on some runs and fails on others with this error:

NoSuchElementError: no such element: Unable to locate element:
{"method":"css selector","selector":"*[id="address-saved"]"}

The confirmation usually appears within 200 milliseconds and sometimes takes 250. The test waits a fixed 200 milliseconds. Selenium's implicit wait is 0 unless the test sets one, so findElement then looks once and fails when the message is late. That makes it a flaky test.

The agent replaces the fixed wait and the lookup with explicit waits. Each explicit wait checks again until its condition is true or 5 seconds pass:

const saved = await driver.wait(
  until.elementLocated(By.id("address-saved")), 5000);
await driver.wait(until.elementIsVisible(saved), 5000);

In Playwright and Cypress, the assertion itself retries, so the same check needs no separate wait. This example is simplified. A real checkout test would also check the cart, e.g. that it still holds 2 items after the address changes.

What changes when a coding agent writes the code?

When a browser test fails, a coding agent can loosen it instead of fixing the app. In Playwright and Cypress, { force: true } on a click skips the checks that the button is visible and not covered, e.g. by a banner, so the click goes through.

An agent can also drive a browser without writing a test, e.g. through Playwright MCP. That session checks the page once. A test file keeps the check, so it can run again on the next change without the agent.

For tests an agent runs, a team can compare frameworks on these points:

• One command. The tests run headless from one command, e.g. npx playwright test, and the run fails if a test fails.
• Built-in waiting. The framework waits for elements and retries assertions, so the agent does not write its own waits.
• Readable failures. The output names the expected and received values in plain text.
• Saved evidence. The runner keeps a screenshot or a trace of each failure for a reviewer.
• A familiar language. The tests use a language the team already reviews.

A reviewer checks each agent edit to retry, timeout, or force settings, because each one can turn a failure into a pass.

What are the limits of browser test frameworks?

A passing browser test run still leaves gaps:

• Only what the test checks. A test checks only the results its assertions name. Playwright can compare the page with an approved screenshot, a form of snapshot testing, but that check can also fail on harmless changes.
• Test browsers, not customers' devices. Tests run in the browsers on the test machine, not on customers' phones. Playwright's device emulation sets a phone's screen size and user agent in a desktop browser engine, which is not a real phone. Its WebKit build is not the Safari app, and its Android support is experimental.
• Speed and timing. Each test needs the running app and a browser, so browser tests run far slower than tests of single functions and fail more often on timing.
• Upkeep. Locators break when the page's markup changes. A self-healing test repairs a broken locator on its own, and that repair can hide a real change.

A passing run shows that the tested journeys worked in the tested browsers, which is not the same as complete coverage.

How do Playwright, Selenium, and Cypress differ?

Playwright, Selenium, and Cypress do the same job, and they differ in how they reach the browser and what they include:

No framework in the table fits every team, and other open-source frameworks exist. Playwright fits teams that want browser builds and built-in waiting from one project. Selenium fits teams that write tests in a language the others lack, e.g. Ruby, or that must drive the Safari app. Cypress fits JavaScript teams that want the test code inside the browser, where it can reach the app's own objects directly.

FAQs

Is Playwright better than Selenium?

Playwright and Selenium suit different teams, so neither is better in general. Playwright includes browser builds and a test runner for JavaScript and TypeScript, and it waits for each element to be ready before it acts. Selenium supports Ruby, which Playwright does not, and drives the Safari app through the W3C WebDriver standard.

Can Playwright test mobile browsers?

Playwright tests mobile browsers mainly through emulation, which gives a desktop browser engine a phone's screen size and user agent. Emulation is not a real phone, and Playwright does not drive the Safari app. Its Android support is experimental.

Does a coding agent need a browser test framework?

A coding agent can try a change in a browser once without writing test code, e.g. through Playwright MCP. A test file in a browser test framework keeps the check, so the same check can run again on each later change without the agent.

Is Cypress open source?

Cypress is open source, and its code is published under the MIT License. Its test runner is included, and its tests run inside the browser in JavaScript or TypeScript. Playwright and Selenium are open source too, under the Apache License 2.0.