cypress-coverage/coverage-final.json is empty object when running cypress-and-jest

I'm following Gleb Bahmutov's instructions for getting combined code coverage reports from cypress and jest (https://github.com/bahmutov/cypress-and-jest), and ran into this issue. Gleb asked me to post the issue here.

When I run "npm t" on my own project, my jest tests display an accurate coverage report, the cypress tests all properly pass, and .nyc-output/out.json file contains a lot of entries for my source files.

However, cypress-coverage/coverage-final.json only contains "{}".

Two things I'm doing a bit differently that I think might have an effect:

(1) I already had cypress running so I did not run "npx @bahmutov/cly init". However, I did check the contents of Gleb's cypress/(plugins|support)/index.js, jest.config.js and package.json to make sure I got everything from them. In package.json, I have these slightly-different versions:

"@cypress/code-coverage": "^3.2.1",
"babel-plugin-istanbul": "^6.0.0",
"check-code-coverage": "^1.0.1",
"cypress": "^4.4.0",
"jest": "^25.3.0",

(2) Under my main project folder, I divided the code into two subfolders, each with its own package.json, because I want the tests to be in a separate subproject than the source (to make it easier to distribute the source without the tests):

project
├── source
│ ├── index.js
│ ├── lib/
│ ├── package.json
│ └── styles.css
└── tests
├── babel.config.js
├── coverage/
├── cypress/
├── cypress-coverage/
├── cypress.json
├── fdnCreate.html
├── fUpdate.html
├── jest.config.js
├── jest-coverage/
├── node_modules/
├── .nyc_output/
├── package.json
├── package-lock.json
├── reports/
└── _ _ tests_ _/

I'll greatly appreciate any suggestions!

[bahmutov]

1. Can you run tests with DEBUG=code-coverage to see what is says when generating the report https://github.com/cypress-io/code-coverage#debugging?
2. Can you paste a few first lines from .nyc_output/out.json so we see the paths there?

debug-npm-t.txt is the result of running DEBUG=code-coverage npm t

Note that "OpplaudSaas" is the actual name of the project, "OpplaudDial" is the name of the source code subproject and "OpplaudDialTests" is the name of the tests subproject.

Here is the complete out.json: out.json.txt

Thank you, again, for taking the time to address this!

[bahmutov]

Can you please install code-coverage plugin v3.3.0 and run JUST the Cypress test with DEBUG=code-coverage npm run test:cy? I don't see any debug logs in the file you have sent (the debug logs are written into STDERR, maybe you are not redirecting it)

Sorry, I'm not sure where to redirect STDERR. I just:

• updated to code-coverage 3.3.0
• changed the package.json script to: ``"test:cy": "cypress run 2>&1",
• ran: ``$ DEBUG=code-coverage npm run test:cy 2>&1 > debug-test-cy.txt

debug-test-cy.txt is the result. I don't see anything that looks like a debug message(?).

I appreciate your help!

[bahmutov]

Hmm what is your support and plugins files? When you use cypress open does it show code coverage messages after each test?

…

Sent from my iPhone

On Apr 18, 2020, at 12:35, Andrew Andrews ***@***.***> wrote:  Sorry, I'm not sure where to redirect STDERR. I just: updated to code-coverage 3.3.0 changed the package.json script to: ``"test:cy": "cypress run 2>&1", ran: ``$ DEBUG=code-coverage npm run test:cy 2>&1 > debug-test-cy.txt debug-test-cy.txt is the result. I don't see anything that looks like a debug message(?). I appreciate your help! — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

Plugins & Support:

plugins-index.js.txt
support-commands.js.txt
support-index.js.txt

Here is the result of running both spec files in cypress open:

debug-cy-open.txt

Thank you kindly,
Andrew

I should mention that, after the above, out.json was an empty object: {}

[bahmutov]

@Andrew-Opplaud how do you instrument your application when running Cypress tests? https://github.com/cypress-io/code-coverage#instrument-your-application

Thanks for your reply, Gleb. My cypress/plugins/index.js includes:

on('file:preprocessor', require('@cypress/code-coverage/use-browserify-istanbul'));

Based on this I was under the impression that would work. Am I mistaken?

I checked and node_modules/@cypress/code-coverage/use-browserify-istanbul.js exists.

I'm not using Babel (except for what Jest does automatically), but I also tried creating .babelrc containing:

{ "plugins": ["istanbul"] }

and added:

on('file:preprocessor', require('@cypress/code-coverage/use-babelrc'));

to plugins/index.js. When I ran npm run cy:test, out.json was still an empty object and the coverage report index.html was empty (except for the boilerplate, of course).

I tried changing the order of the on('file:preprocessor',...) lines in case that mattered, but it didn't. I also tried commenting-out the use-browserify-istanbul preprocessor when including use-babelrc, but the results were the same.

I appreciate your continued assistance. What should I try next?

[bahmutov]

Gotcha - so the "Alternative" way still applies only to unit test instrumentation. Let me add it to README, but in essence:

if your code does cy.visit(index.html) then anything index.html is loading has to be instrumented by YOU.

if your code is loaded directly from spec file, like this - Cypress can instrument it for you, this is called "unit test instrumentation".

import {add} from '../../src/utils'
it('works', () => {
  expect(add(2,3)).to.equal(5)
})

Thus in your case, YOUR web application was not instrumented, and thus output code coverage is empty.

[bahmutov]

Can you take a look at https://on.cypress.io/code-coverage#Instrumenting-code we give more advice there

Thanks for the explanation. I had hoped that combining Jest and Cypress (as per your cypress-and-jest example) automated the instrumentation for me, but I see now that I didn't notice the subtlety that the instrumentation was automated for Cypress unit tests, not the E2E tests.

So firstly, it sounds like Jest isn't really helping me in this situation: if I use Cypress for the unit tests as well as the E2E tests, then I can take Jest out of the equation. Is that correct?

Secondly, I've added a script to package.json:

`"nyc": "cd ~/OpplaudSaas && nyc instrument --compact=false OpplaudDial OpplaudDialTests/instrumented",`

Now, when I run npm run nyc, it quickly generates the files in the OpplaudDialTests\instrumented folder that correspond to the files in OpplaudDial (project source) folder but does not generate an .nyc_output/out.json file.

Is this to be expected?

Also, I've modified the test:cy script to:

`"test:cy": "npm run nyc && cypress run",`

and modified the import statements in my <script> elements to reference the ./instrumented/** versions.

Now, when I run npm run test:cy, it runs all of the tests and they pass, and I get
out.json.txt with references back to the original sources instead of the instrumented versions.

Is this also to be expected?

Lastly, when I look in cypress-coverage\lcov-report\index.html, it's still essentially empty. :(

I feel like I'm getting closer... I hope I'm getting closer!?

Thanks again for any advice you can provide!

[bahmutov]

Can you please run with DEBUG=code-coverage and have the logs? With version 3.4.1 of the plugin.

Done... this is interesting: after running, there is now a coverage folder instead of a cypress-coverage folder, even though package.json still contains "nyc":{"report-dir": "cypress-coverage"}.

That probably explains the error messages in the output.

out.json looks the same as usual.

Thanks again!