How to measure code coverage of a Next.js app via end-to-end Playwright tests?

[anishkny]

Hello,

I am interested in measuring code coverage of a Next.js app started with npx next dev. My setup is:

$ cat pages/index.js
function HomePage() {
  return <div>Welcome to Next.js!</div>;
}
export default HomePage;

$ cat package.json`
{
  "dependencies": {
    "next": "^11.1.0",
    "react": "^17.0.2",
    "react-dom": "^17.0.2"
  }
}

$ npx nyc --reporter html npm run dev

# ... exercise the app in browser

This results in coverage reports starting with:

/*
 * ATTENTION: An "eval-source-map" devtool has been used.
 * This devtool is neither made for production nor for readable output files.
 * It uses "eval()" calls to create a separate source file with attached SourceMaps in the browser devtools.
 * If you are trying to read the output file, select a different devtool (https://webpack.js.org/configuration/devtool/)
 * or disable the default devtool with "devtool: false".
 * If you are looking for production-ready output files, see mode: "production" (https://webpack.js.org/configuration/mode/).
 */

See coverage.zip

What is the correct way to measure code coverage for a Next.js app?

[thibautsabot]

Hello !
When you mention "code coverage" does it mean the coverage of your unit tests ?
If so, Next.js behave exactly the same as a basic React app.

You can follow https://nextjs.org/docs/testing#jest-and-react-testing-library to get instruction on how to set up Jest with Next.js
Jest provides a nice --coverage option for this usecase.

I don't think running your coverage against the npm run dev is what you want.
The coverage is generated by a test runner like jest, cypress, ... you can't measure the coverage of a generated output.

[anishkny]

@thibautsabot thanks for the reply.

I realize I should have been clearer in my question. We are running automated end-to-end tests using Playwright (a browser automation framework) not unit tests and want to measure code coverage as a result of running these end-to-end tests.

Note: I have updated the title and description to reflect this.

[anishkny]

Upon further investigation, mxschmitt/playwright-test-coverage seems to demonstrate what I need.

The key is to extend the base Playwright test runner to capture window.__coverage__ and serialize it at the appropriate time. See baseFixtures.ts.

UPDATE Sep 6, 21: I have published playwright-test-coverage npm package which can be used as follows instead of copy-pasting the solution from the repo:

npm i -D playwright-test-coverage

const { test, expect } = require("playwright-test-coverage");

// Use test and expect as usual
test('basic test', async ({ page }) => {
  // Test code
});

Ideally, this should be an option in Playwright itself.

[IonelLupu]

How can we do code coverage without a babel.config.js file? It seems Nexjts recommends not to use it and instead use "swc" instead.

[kylemh]

@anishkny 's accepted answer has been non-viable since Next.js v14 afaict

@feedthejim, I'm sure you're crazy busy, but if you're ever interested... I'd love to walk you through what the dev story of setting up code coverage for tools like Playwright and Cypress are today. It's not a great story, and I'm hoping it could lead to the Next.js team prioritizing some effort to making this easier.