E2E tests with Cypress do not use .env.test for env variables

[lcmen]

Version

3.0.0-beta.9

Reproduction link

https://github.com/lowski/fakturownia/tree/managing-clients - it's an app that I use for testing different techniques for Vue.js app but setup is the default one, created by Vue-cli.

Steps to reproduce

1. Create .env.test file with some env variable, i.e:

VUE_APP_API_URL=http://localhost:8081

2. Try to reference that variable inside any of your client-side JavaScript files, i.e.:

console.log(process.env.VUE_APP_API_URL)

What is expected?

It is expected to print http://localhost:8081 when tests are run with Cypress

What is actually happening?

It's printing undefined

---

I tried to pass --mode test to e2e command like this:

vue-cli-service test:e2e --mode test

But it throws error that HMR is disabled and test does not run.

[LinusBorg]

from the docs of that plugin:

The command automatically starts a server in production mode to run the e2e tests against. If you want to run the tests multiple times without having to restart the server every time, you can start the server with vue-cli-service serve --mode production in one terminal, and then run e2e tests against that server using the --url option.

From https://github.com/vuejs/vue-cli/blob/dev/docs/env.md#modes

Mode is an important concept in Vue CLI projects. By default, there are three modes in a Vue CLI project:

• development is used by vue-cli-service serve
• production is used by vue-cli-service build and vue-cli-service test:e2e
• test is used by vue-cli-service test:unit

So you can either provide a .env.production file, or, if you rather want to use a different mode, do this:

# .env.e2e
NODE_ENV=production
VUE_APP_API_URL=http://localhost:8081

vue-cli-service test:e2e --mode e2e

The important part is to set NODE_ENV to production so the backend config is setup like a production build, while the environment variables can be different.

[lcmen]

@LinusBorg thanks for help, it fixed the problem with HMR being disabled but unfortunately it does not work for providing correct env variables to the application's code - VUE_APP_API_URL is still undefined.

I believe the problem is related to building application with production env (NODE_ENV=production). When you do that it will look into the production files (.env.production and .env.production.local) for the variables. That mens the application's code which is being tested with Cypress is built with variables from those files instead of e2e ones (.env.e2e and .env.e2e.local).

Can we reopen it as the fix from above does not solve the original issue?

[LinusBorg]

I believe the problem is related to building application with production env (NODE_ENV=production). When you do that it will look into the production files (.env.production and .env.production.local) for the variables.

No, that 's not how it works. the env file to load is determined by the --mode, as explained in the documentation bout env files that I linked in my previous comment. NODE_ENV not the same thin as "the mode", and is only used within cli-service to determine what features to include into the config, not to determine which env file to load.

this is also why you have to include NODE_ENV in a custom environment's that are not development, production or test.

If that does not work in this case, it might be a bug. I will re-open this issue - if you provide a repository with my fix applied and still the test is still failing. You should be able to do that with a commit to the repository you linked to in your OP., but I would prefer a clean setup that doesn't contain custom code that's not required to reproduce the problem.

[lcmen]

No, that 's not how it works. the env file to load is determined by the --mode, as explained in the documentation bout env files that I linked in my previous comment. NODE_ENV not the same thin as "the mode", and is only used within cli-service to determine what features to include into the config, not to determine which env file to load.

Thanks for the explanation. Then it's a bug. I've updated my repo with solution you suggested previously - https://github.com/lowski/fakturownia/tree/managing-clients
Remember to start test server (npm run test:server) before running e2e tests.

Btw. Is there a reason why import xyz from '@/path/in/src' is not supported in e2e tests? For mitigate this, I've added src directory to NODE_PATH in e2e mode.

[LinusBorg]

Where in your project so you reference the env variable?

@/ doesn't work inside the spec files because it's a webpack alias, and the specs are not handled by webpack, they are handled by cypress.

[lcmen]

In src/lib/axios I reference this env variable.

[lcmen]

@LinusBorg I think the problem might be caused by Cypress using browserify by default. Seems that it looks for .env.cypress file with env variable by default (I believe I've found this info somewhere in the docs).

I think it make sense to use Webpack preprocessor instead of Browserify so we can use the same mechanism to load envs as for the application's code (it would also allow to provide alias for @).

To use Webpack instead of Browserify, I've added the following code:

// test/e2e/plugins/index.js
const path = require("path");
const webpack = require("@cypress/webpack-preprocessor");

module.exports = (on, config) => {
  const options = Object.assign({}, webpack.defaultOptions);
  options.webpackOptions.resolve = {
    alias: {
      // Add `@` alias to be compatible with source config
      "@": path.resolve(path.join(__dirname, "../../..", "src"))
    }
  };
  // Ignore .babelrc from the project root so it won't complain about vue preset
  options.webpackOptions.module.rules[0].use[0].options.babelrc = false;

  on("file:preprocessor", webpack(options));
  return Object.assign({}, config, {
    fixturesFolder: "tests/e2e/fixtures",
    integrationFolder: "tests/e2e/specs",
    screenshotsFolder: "tests/e2e/screenshots",
    videosFolder: "tests/e2e/videos",
    supportFile: "tests/e2e/support/index.js"
  });
};

This way I'm able to use import xyz from '@/some/path/in/src'.

I've pushed that code here https://github.com/lowski/fakturownia/tree/cypress-with-webpack if you would like to take a look.

[LinusBorg]

I just took a look at the code (can't run it right now) and frankly am a bit confused.

I don't see what we need webpack for - the client code is packaged and served by the devserver, so what is webpack required for? Just to use the alias in spec files? Or am I missing something fundamental about Cypress (not an expert)?

Seems that it looks for .env.cypress file with env variable by default (I believe I've found this info somewhere in the docs).

That would make sense for env variables that should be available in the cypress setup/process - but I understood that you want to change an env variable within the client side code compiled by npm run serve?

[lcmen]

I just took a look at the code (can't run it right now) and frankly am a bit confused.

What is confusing for you?

Let's get back to the beginning - I wanted to import some code from my application's to my tests - specifically axios setup - so I could create same data on the test server for testing (app and test uses the same test server running on port 8081). I thought that .env.test is shared between my application's code and my test's code (maybe I'm biased by coming from Ember that would make sense) and that's how I understood the documentation.

Apparently it looks like Cypress by default uses some different env file so that's where the confusion starts. To me it's either an issue with documentation (which does not mention that test code uses different env file) or with Cypress setup in vue-cli.

@LinusBorg I hope it's clear now for you. If you have any more questions feel free to ask.

I don't see what we need webpack for - the client code is packaged and served by the devserver, so what is webpack required for? Just to use the alias in spec files?

No just for alias but also for loading the correct env file and exposing its values to the test code, the same way as it's done on the application side.

[LinusBorg]

Thanks, it's clear now. Your initial explanation only referred to "client side files", not to the fact that you wanted to actually import them into your e2e tests (which is not that common in e2e tests, compared to unit tests where you do it in each spec, obviously). Your suspicion is right, cypress and the webpack-dev-server don't share env variables (that fact was too obvious to me as someone involved in the project and in turn the source of my confusion).

Cypress doesn't load .env files at all, it uses .json files to define environment variables. (https://docs.cypress.io/guides/guides/environment-variables.html#)

And since it's not running within our node process that we started with vu-cli-service, it also can't share the env variables that cli-service loads from the .env files.

So we should talk about wether it's worth to expose the contents of .env files to cypress automatically, or if we should provide some minimal documentation about how to set env variables for cypress individually.

[yyx990803]

I think the easiest way is using CYPRESS_ prefixed env variables as documented here: https://docs.cypress.io/guides/guides/environment-variables.html#Option-3-CYPRESS

We can probably mention this in the plugin's README.