Symfony: `.env.local` and `.env.test` are ignored

[ThomasLandauer]

Starting in November 2018, Symfony has changed the way .env files are working: https://symfony.com/doc/current/configuration.html#the-env-file-environment-variables

These changes are not realized in Codeception.

To assert this, I created a controller which just outputs an environment variable:

return new Response(getenv('MAILER_URL'));`

When I access the url with codeception (codecept_debug($I->grabPageSource());), I always get what I have in .env (should be .env.local or .env.test).

However, when I open the page in the browser in DEV environment, I get the content of .env.dev, when in TEST environment, .env.test; this is the expected behavior.

Details

• Codeception version: 2.5.x-dev fc6deb8
• Symfony version: 4.2.1

[Naktibalda]

@ThomasLandauer Can you fix it?

[leroy0211]

Same here

[alexkunin]

I have similar issue, it happens because Codeception neither loads environment directly nor uses Symfony's way to do that (specifically, does not include config/bootstrap.php).

My solution is the following snippet in codeception.yml:

params:
  - .env.test

Maybe it will be better to modify Symfony module to check version and config/env file layout and automatically load config/bootstrap.php. Though, that file could contain custom code which might not be desired in testing environment. So, the fix above is a bit awkward but also least risky for now?

[c33s]

the main problem is, that config/bootstrap.php is not loaded and also that phpunit.xml or phpunit.xml.dist also are not considered.

my phpunit.xml.dist:

...
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="http://schema.phpunit.de/6.5/phpunit.xsd"
         backupGlobals="false"
         colors="true"
         bootstrap="config/bootstrap.php"
>
    <php>
        <ini name="error_reporting" value="-1" />
        <server name="APP_ENV" value="test" force="true" />
...

so phpunit normally loads config/bootstrap.php and also sets APP_ENV=test. so a solution could be to parse the phpunit.xml* and respect the content.

my workaround is to rebuild the behavior of phpunit.xml.dist with codeception.

1. remove all params from codeception.yml
2. load symfonys config/bootstrap.php in codeceptions tests/_bootstrap.php

ad 1.

#params:
#    - .env.test # get params from .env.test file
#    - env # to get params from environment vars (only real env)
#    - .env # get params from .env file

ad 2.
tests/_bootstrap.php:

// force test environment
$_ENV['APP_ENV'] = 'test';
// load the symfony bootstrap file to ensure correct dotenv handling.
require dirname(__DIR__).'/config/bootstrap.php';

ad 3.

Add this to your codeception.yml to load the new tests/_bootstrap.php you just created - see https://codeception.com/docs/reference/Configuration#Global-Configuration:

    bootstrap: _bootstrap.php

• Codeception 2.5.1
• Symfony 3.4 with symfony 4.x directory structure and a symfony 4.x bootstrap.php file

edit: updated comment with comment from @ThomasLandauer https://github.com/Codeception/Codeception/issues/5411#issuecomment-585279655

[ThomasLandauer]

@c33s solution works for me (Symfony 4.4) - there's just one step missing:

3. Add this to your codeception.yml to load the new tests/_bootstrap.php you just created - see https://codeception.com/docs/reference/Configuration#Global-Configuration:

   bootstrap: bootstrap.php

[Naktibalda]

I'm using Symfony 5 in my current project and I had this issue too.

Adding env.test file to params section worked for me, but it would definitely be better if Symfony module did it automatically.

Including config/bootstrap in _initialize method works too, but it is a breaking change, so it should be released as 2.0.0 of Symfony module.

An alternative approach is to add .env.test to codeception.yaml in recipe: https://github.com/symfony/recipes-contrib/blob/master/codeception/codeception/2.3/codeception.yaml
The only reason .env is currently loaded automatically is that it is set by that recipe file.

Currently I prefer modifying recipe file.

[Naktibalda]

So I wanted to commit my code and realized that env.test must be committed to git, but my secrets are there.

Comment at the top of .env says that

# In all environments, the following files are loaded if they exist,
# the latter taking precedence over the former:
#
#  * .env                contains default values for the environment variables needed by the app
#  * .env.local          uncommitted file with local overrides
#  * .env.$APP_ENV       committed environment-specific defaults
#  * .env.$APP_ENV.local uncommitted environment-specific overrides

Comment above Dotenv::loadEnv method says that

• .env.local is always ignored in test env because tests should produce the same results for everyone.

Secrets must go to .env.test.local file, right?
Then .env.test.local must be enabled in params section and if the file doesn't exist codeception run fails with error that Params file .../.env.test.local not found.

Updating recipe looks like bad option now and it would be better to include bootstrap.php

[ThomasLandauer]

Maybe a stupid question, but which secrets do you have in test environment?

[dant89]

Same issue here, isn't possible to pass in testing credentials without hacking the config.

[c33s]

why hack the config? can you explain the problem? do you have "real" secrets in your tests? i have dummy secrets for my tests and the are in .env.test and commited inside the repo to ensure tests are running everywhere. so what "secrets" are you trying to store? @ThomasLandauer & @dant89

[dant89]

@c33s Yes we do have "real" secrets because we test production APIs and third party API / bundle integrations periodically, I don't think that's the point though... .env.test.local should be read over .env.test. Turning this into a discussion about why it's okay to put "real" secrets into git repositories isn't helpful or the right solution.

[callmebob2016]

I tried the approach suggested by @c33s but it doesn't seem to work for me. Controller always uses .env instead of switching to the .env.test. I am using Symfony5.

I removed all params: from codeception.yml and added bootstrap: bootstrap.php. I also added $_ENV['APP_ENV'] = 'test'; to tests/bootstrap.php. var_dump still shows values from .env.

Any idea what I may be missing ?

[c33s]

@dant89 ok, i get it, you have real secrets in your file, so .env.test.local (which is not in the repo) is the right location for it. what is the problem/task you want to archive? if you put the secrets in the .env.test.local file or maybe even in your real environment, they are "safe" there. i use gitlab, which has its own CI, where i have a gitlab-ci.yml where i can define non-secret env variables for the docker container and in the gitlab gui i can define secret environment variables that are additionally set for the docker container where the tests run. if you have server, which is doing the production "tests" (calling external apis to check them periodically has nothing to do with testing for me, this is monitoring). than you can even deploy a .env.test.local directly on the server.

my question would be, what are you trying to archive?

[c33s]

@callmebob2016 do you have a minimal example for me? for me it sounds really stange, if you define $_ENV['APP_ENV']=test that your app is started in dev/prod env.
i think it's a dotenv or php config problem,

symfony initiates the dotenv class in bootstrap.php:

---snip---
if (is_array($env = @include dirname(__DIR__).'/.env.local.php') && (!isset($env['APP_ENV']) || ($_SERVER['APP_ENV'] ?? $_ENV['APP_ENV'] ?? $env['APP_ENV']) === $env['APP_ENV'])) {
    foreach ($env as $k => $v) {
        $_ENV[$k] = $_ENV[$k] ?? (isset($_SERVER[$k]) && 0 !== strpos($k, 'HTTP_') ? $_SERVER[$k] : $v);
    }
} elseif (!class_exists(Dotenv::class)) {
    throw new RuntimeException('Please run "composer require symfony/dotenv" to load the ".env" files configuring the application.');
} else {
    // load all the .env files
    (new Dotenv(false))->loadEnv(dirname(__DIR__).'/.env');
}
---snip---

first check if the dotenv is instanced with false or true the default parameter was changed from true to false as far as i can remember.
use your debugger or just use dump to see if dotenv ist even loaded. check the content of APP_ENV.

or it can have something to do with the phpconfig variables_order = EGPCS see https://www.php.net/manual/en/ini.core.php
can be that your env variables are not set.

general note: stuff defined in .env is always loaded if not overwritten from .env.test or .env.test.local (in this order)

how the dotenv component has changed and how it's working:
https://symfony.com/blog/new-in-symfony-4-2-define-env-vars-per-environment
https://symfony.com/doc/current/configuration/dot-env-changes.html

[c33s]

btw i am online in slack (symfony & codeception)