Photo by Ferenc Almasi on Unsplash

Microsoft Playwright for cross-browser E2E tests

When asked what libraries they use for E2E tests, people generally say that they use Puppeteer, Cypress, and Selenium. For some reason haven’t really heard of Microsoft Playwright.

Puppeteer, Cypress, and Selenium all have inherit limitations. A common limitation of all three are the lack of cross-browser support. All three support Chromium and some have support for Firefox as well. The lack of Safari support is glaring. With around 20% of all browser users using Safari, E2E testing on Safari is no longer optional.

Microsoft Playwright supports all 3 browsers. With the same APIs! The way they’ve done this is by monkey-patching the underlying Webkit engine to support all the APIs they need. And it works excellently. So without further ado, let’s get started.

First off we’ll need to create a new npm project.

mkdir playwright-e2e && cd playwright-e2e && npm init -y

Now to install Playwright with its built-in test runner.

npm i -D @playwright/test
npx playwright install

Now just to make sure everything is up and running correctly create a folder called tests and inside that folder create a file called index.spec.js

const { test, expect } = require("@playwright/test");
test('basic test', async ({ page }) => {
  await page.goto('https://playwright.dev/');
  const title = page.locator('.navbar__inner .navbar__title');
  await expect(title).toHaveText('Playwright');
});

And now in order to run it you can use one of the following commands

# Run all tests using Chromium (headlessly)
npx playwright test
-------------------------
# Run all tests using Chromium (headed)
npx playwright test --headed
-------------------------
# Run all tests using Safari
npx playwright test --browser=webkit
-------------------------
# Run all tests using all browsers
npx playwright test --browser=all

You should see output that looks similar to this:

Amazing!

Before we continue, I’d like to insert an opinion here now. There are multiple ways of approaching E2E testing. One method is by manually verifying that all the elements that are supposed to be on the page, are actually there, using manual assertions. A downside of this method is that it takes time. A lot of time.

I personally prefer the visual comparison method. The flow essentially works as follows:

• Run through all tests once and create the baseline screenshots
• All subsequent test runs will now compare their screenshots to the baseline one.

The reason I like this method so much is because you get the benefits of manual assertions for free. By definition, if the screenshots match the baseline, the element definitely appears (or doesn’t appear) on the page.

So let’s get into it.

A basic setup for a test looks like this now.

const { test, expect } = require("@playwright/test");
test("basic test", async ({ page }) => {
  await page.goto("https://playwright.dev/");
  expect(await page.screenshot()).toMatchSnapshot("get-started.png");
});

For the first run we need to first create the baseline screenshots. You can do this by running:

npx playwright test --update-snapshots

After this runs, you should now see the following in your project directory structure

The baseline screenshots will appear in a directory next to each .spec.js file.

Now you can run the tests normally using any of the aforementioned commands (like npx playwright test ).

Another benefit of this approach is that in the event that the screenshots don’t match, you get an image with a visual diff. Let’s say that for some reason, your website’s page starts scrolling down a bit on load. The image diff you’ll see may look something like this:

Great!

Something to note is that if you have dynamic content (like a countdown timer) this approach of E2E testing will not work.

Thanks for reading.

For more content like this you can find me on Twitter: JacobBeckerman