Coverage not collected when using async import

[penx]

Logs and screenshots
Please provide debug logs by running Cypress from the terminal with DEBUG=code-coverage environment variable set, see the Debugging section of the README file.

Versions

• What is this plugin's version? 3.8.3
• What is Cypress version? 5.6.0
• What is your operating system? Mac OS X 10.15.7
• What is the shell? zsh
• What is the Node version? 12.14.1
• What is the NPM version? 6.13.4
• How do you instrument your application? @cypress/instrument-cra@1.4.0
• When running tests, if you open the web application in regular browser, and open DevTools, do you see window.__coverage__ object? Can you paste a screenshot?
• Is there .nyc_output folder? Is there .nyc_output/out.json file. Is it empty? Can you paste at least part of it so we can see the keys and file paths?

{
  "/storybook-code-coverage/src/utilities/useToggle.js": {
    "path": "/storybook-code-coverage/src/utilities/useToggle.js",
    "statementMap": {
      "0": {
        "start": {
          "line": 5,
          "column": 25
        },
        "end": {
          "line": 13,
          "column": 1
        }

• Do you have any custom NYC settings in package.json (nyc object) or in other NYC config files

  "nyc": {
    "report-dir": "coverage/integration",
    "reporter": [
      "text",
      "json",
      "lcov"
    ],
    "all": true,
    "include": [
      "src/**/*.js"
    ],
    "exclude": [
      "**/*.test.js",
      "**/test.js",
      "**/*.stories.js",
      "**/stories.js",
      "src/setupTests.js"
    ]
  },

• Do you run Cypress tests in a Docker container? No

Describe the bug

If you use async import in your code, coverage is not correctly collected from the files that are loaded async.

e.g.

If I have this code:

const Users = React.lazy(() => import('./Users'));

If I run 2 specs:

Spec 1 doesn't load the Users component
Spec 2 does load the users component

Overall coverage says that the Users component was not covered, when in fact it was.

If you swap the order of the specs (rename 1-spec to 3-spec), then Users is correctly shown as covered.

If you only run 2-spec (delete 1-spec) then Users is correctly shown as covered.

Link to the repo

This repo:

https://github.com/penx/storybook-code-coverage/tree/lazy-loading-bug

• Make sure you are on the lazy-loading-bug branch
• checkout, and run: yarn; yarn coverage:clean; yarn coverage:integration
• Note that in output, the src/application/Users has no coverage, even though 2-spec.js tests it.

[bahmutov]

It is weird, since it has the counters, but not the original function or statement maps

[bahmutov]

Ok, so this is happening because:

• we want to instrument users/index.js which is not in the original coverage object (since we are on page 1)
• we need that file, so we insert a placeholder record with empty info for this file
• we save the coverage JSON
• next we get the test to go to the /users page and our coverage has the /users/index.js actual data. It has the counter and the maps to the source
• we load the previous coverage JSON (with placeholder object) and merge it with the good info.

And at the last step the things go wrong: the istanbul's coverage merge logic does NOT merge correctly the statement, function, and branch maps, instead just merging the counters.

[bahmutov]

🎉 This issue has been resolved in version 3.8.8 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀

[penx]

great, thanks @bahmutov ! I'll try check it out soon though it may not be until January