Skip to content

Symfony 4.4 secrets management system #12958

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jan 27, 2020
Merged

Symfony 4.4 secrets management system #12958

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
82d2058
Add the secret documentation
jderusse Apr 10, 2019
75e5ae6
finishing the secrets documentation
weaverryan Jan 18, 2020
cd066cc
tweaks thanks to review
weaverryan Jan 26, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions best_practices.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,12 @@ application behavior.
:ref:`Use env vars in your project <config-env-vars>` to define these options
and create multiple ``.env`` files to :ref:`configure env vars per environment <config-dot-env>`.

Use Secret for Sensitive Information
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

When your application has sensitive configuration - like an API key - you should
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would use a database password instead of an api key for the example
99% of project have a db :)
Wdyt?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see your point. But on some systems (Heroku, SymfonyCloud), the database credentials are provided to you by your platform and are not something you need to store in a vault.

store those securely via :doc:`secrets </configuration/secrets>`.

Use Parameters for Application Configuration
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Expand Down
214 changes: 109 additions & 105 deletions configuration.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -436,14 +436,14 @@ Configuration Based on Environment Variables

Using `environment variables`_ (or "env vars" for short) is a common practice to
configure options that depend on where the application is run (e.g. the database
credentials are usually different in production and in your local machine).
credentials are usually different in production versus your local machine). If
the values are sensitive, you can even :doc:`encrypt them as secrets </configuration/secrets>`.

Instead of defining those as regular options, you can define them as environment
variables and reference them in the configuration files using the special syntax
You can reference environment variables using the special syntax
``%env(ENV_VAR_NAME)%``. The values of these options are resolved at runtime
(only once per request, to not impact performance).

This example shows how to configure the database connection using an env var:
This example shows how you could configure the database connection using an env var:

.. configuration-block::

Expand Down Expand Up @@ -485,164 +485,168 @@ This example shows how to configure the database connection using an env var:
]
]);

The next step is to define the value of those env vars in your shell, your web
server, etc. This is explained in the following sections, but to protect your
application from undefined env vars, you can give them a default value using the
``.env`` file:

.. code-block:: bash

# .env
DATABASE_URL=sqlite:///%kernel.project_dir%/var/data.db

.. seealso::

The values of env vars can only be strings, but Symfony includes some
:doc:`env var processors </configuration/env_var_processors>` to transform
their contents (e.g. to turn a string value into an integer).

In order to define the actual values of env vars, Symfony proposes different
solutions depending if the application is running in production or in your local
development machine.

Independent from the way you set environment variables, you may need to run the
``debug:container`` command with the ``--env-vars`` option to verify that they
are defined and have the expected values:
To define the value of an env var, you have several options:

.. code-block:: terminal
* :ref:`Add the value to a .env file <config-dot-env>`;
* :ref:`Encrypt the value as a secret <configuration-secrets>`;
* Set the value as a real environment variable in your shell or your web server.

$ php bin/console debug:container --env-vars

---------------- ----------------- ---------------------------------------------
Name Default value Real value
---------------- ----------------- ---------------------------------------------
APP_SECRET n/a "471a62e2d601a8952deb186e44186cb3"
FOO "[1, "2.5", 3]" n/a
BAR null n/a
---------------- ----------------- ---------------------------------------------
.. tip::

# you can also filter the list of env vars by name:
$ php bin/console debug:container --env-vars foo
Some hosts - like SymfonyCloud - offer easy `utilities to manage env vars`_
in production.

# run this command to show all the details for a specific env var:
$ php bin/console debug:container --env-var=FOO
.. caution::

.. versionadded:: 4.3
Beware that dumping the contents of the ``$_SERVER`` and ``$_ENV`` variables
or outputting the ``phpinfo()`` contents will display the values of the
environment variables, exposing sensitive information such as the database
credentials.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even more, they are exposed in the system (for instance, when requesting details about PHP-fpm processes or services). I think we maybe should be more explicit that plaintext sensitive information should not be in an env var

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's true... but only true when users are setting "real" env vars (not using .env), right? In general, I hope we can now (with secrets) really push people to use that system. This warning already is starting to look misplaced to me: we should (in the document in general) be telling people to instantly use the secrets management for sensitive stuff.


The option to debug environment variables was introduced in Symfony 4.3.
The values of the env vars are also exposed in the web interface of the
:doc:`Symfony profiler </profiler>`. In practice this shouldn't be a
problem because the web profiler must **never** be enabled in production.

.. _configuration-env-var-in-dev:
.. _config-dot-env:

Configuring Environment Variables in Development
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Configuring Environment Variables in .env Files
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Instead of defining env vars in your shell or your web server, Symfony proposes
a convenient way of defining them in your local machine based on a file called
``.env`` (with a leading dot) located at the root of your project.
Instead of defining env vars in your shell or your web server, Symfony provides
a convenient way to define them inside a ``.env`` (with a leading dot) file
located at the root of your project.

The ``.env`` file is read and parsed on every request and its env vars are added
to the ``$_ENV`` PHP variable. The existing env vars are never overwritten by
the values defined in ``.env``, so you can combine both.
to the ``$_ENV`` & ``$_SERVER`` PHP variables. Any existing env vars are *never*
overwritten by the values defined in ``.env``, so you can combine both.

This is for example the content of the ``.env`` file to define the value of the
``DATABASE_URL`` env var shown earlier in this article:
For example, to define the ``DATABASE_URL`` env var shown earlier in this article,
you can add:

.. code-block:: bash

# .env
DATABASE_URL="mysql://db_user:db_password@127.0.0.1:3306/db_name"

This file should be committed to your repository and (due to that fact) should
only contain "default" values that are good for local development. This file
should not contain production values.
Copy link
Contributor

@94noni 94noni Jan 19, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would use sensitives values
As on a preproduction, there are also such values

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sensitive value should not be committed and therefore should not be stored in .env file which contains default public values.

I'd rather use secrets to store "pre-prodction" sensitive value and change the secret location (see section configuration) according to an env variable that define the stage.
ie.

framework:
  secrets:
    vault_directory: '%kernel.project_dir%/config/secrets/%kernel.environment%/%app_stage'
parameters:
  app_stage: '%env(STAGE)%'
$ STAGE=PRE-PRODCTION APP_ENV=PROD bin/console secrets:set DATABASE_PASSWORD

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be a nice example to add to the config section after this PR is merged :)


In addition to your own env vars, this ``.env`` file also contains the env vars
defined by the third-party packages installed in your application (they are
added automatically by :ref:`Symfony Flex <symfony-flex>` when installing packages).

.. _configuration-multiple-env-files:

Overriding Environment Values via .env.local
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If you need to override an environment value (e.g. to a different value on your
local machine), you can do that in a ``.env.local`` file:

.. code-block:: bash

# .env.local
DATABASE_URL="mysql://root:@127.0.0.1:3306/my_database_name"

This file should be ignored by git and should *not* be committed to your repository.
Several other ``.env`` files are available to set environment variables in *just*
the right situation:

* ``.env``: defines the default values of the env vars needed by the application;
* ``.env.local``: overrides the default values for all environments but only on
the machine which contains the file. This file should not be committed to the
repository and it's ignored in the ``test`` environment (because tests should
produce the same results for everyone);
* ``.env.<environment>`` (e.g. ``.env.test``): overrides env vars only for one
environment but for all machines (these files *are* committed);
* ``.env.<environment>.local`` (e.g. ``.env.test.local``): defines machine-specific
env var overrides only for one environment. It's similar to ``.env.local``,
but the overrides only apply to one environment.

*Real* environment variables always win over env vars created by any of the
``.env`` files.

The ``.env`` and ``.env.<environment>`` files should be committed to the
repository because they are the same for all developers and machines. However,
the env files ending in ``.local`` (``.env.local`` and ``.env.<environment>.local``)
**should not be committed** because only you will use them. In fact, the
``.gitignore`` file that comes with Symfony prevents them from being committed.

.. caution::

Applications created before November 2018 had a slightly different system,
involving a ``.env.dist`` file. For information about upgrading, see:
:doc:`configuration/dot-env-changes`.

.. _configuration-env-var-in-prod:

Configuring Environment Variables in Production
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In production, the ``.env`` files are also parsed and loaded on each request so
you can add env vars to those already defined in the server. In order to improve
performance, you can run the ``dump-env`` command (available when using
:ref:`Symfony Flex <symfony-flex>` 1.2 or later).
In production, the ``.env`` files are also parsed and loaded on each request. So
the easiest way to define env vars is by deploying a ``.env.local`` file to your
production server(s) with your production values.

This command parses all the ``.env`` files once and compiles their contents into
a new PHP-optimized file called ``.env.local.php``. From that moment, Symfony
will load the parsed file instead of parsing the ``.env`` files again:
To improve performance, you can optionally run the ``dump-env`` command (available
in :ref:`Symfony Flex <symfony-flex>` 1.2 or later):

.. code-block:: terminal

# parses ALL .env files and dumps their final values to .env.local.php
$ composer dump-env prod

After running this command, Symfony will load the ``.env.local.php`` file to
get the environment variables and will not spend time parsing the ``.env`` files.

.. tip::

Update your deployment tools/workflow to run the ``dump-env`` command after
each deploy to improve the application performance.

.. _configuration-env-var-web-server:
.. _configuration-secrets:

Creating ``.env`` files is the easiest way of using env vars in Symfony
applications. However, you can also configure real env vars in your servers and
operating systems.
Encrypting Environment Variables (Secrets)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. tip::

SymfonyCloud, the cloud service optimized for Symfony applications, defines
some `utilities to manage env vars`_ in production.

.. caution::
Instead of defining a real environment variable or adding it to a ``.env`` file,
if the value of a variable is sensitive (e.g. an API key or a database password),
you can encrypt the value using the :doc:`secrets management system </configuration/secrets>`.

Beware that dumping the contents of the ``$_SERVER`` and ``$_ENV`` variables
or outputting the ``phpinfo()`` contents will display the values of the
environment variables, exposing sensitive information such as the database
credentials.

The values of the env vars are also exposed in the web interface of the
:doc:`Symfony profiler </profiler>`. In practice this shouldn't be a
problem because the web profiler must **never** be enabled in production.

.. _configuration-multiple-env-files:

Managing Multiple .env Files
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Listing Environment Variables
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The ``.env`` file defines the default values for all env vars. However, it's
common to override some of those values depending on the environment (e.g. to
use a different database for tests) or depending on the machine (e.g. to use a
different OAuth token on your local machine while developing).
Regardless of how you set environment variables, you can see a full list with
their values by running:

That's why you can define multiple ``.env`` files to override env vars. The
following list shows the files loaded in all environments. The ``.env`` file is
the only mandatory file and each file content overrides the previous one:
.. code-block:: terminal

* ``.env``: defines the default values of the env vars needed by the application;
* ``.env.local``: overrides the default values of env vars for all environments
but only in the machine which contains the file (e.g. your development computer).
This file should not be committed to the repository and it's ignored in the
``test`` environment (because tests should produce the same results for everyone);
* ``.env.<environment>`` (e.g. ``.env.test``): overrides env vars only for some
environment but for all machines;
* ``.env.<environment>.local`` (e.g. ``.env.test.local``): defines machine-specific
env vars overrides only for some environment. It's similar to ``.env.local``,
but the overrides only apply to some particular environment.
$ php bin/console debug:container --env-vars

.. note::
---------------- ----------------- ---------------------------------------------
Name Default value Real value
---------------- ----------------- ---------------------------------------------
APP_SECRET n/a "471a62e2d601a8952deb186e44186cb3"
FOO "[1, "2.5", 3]" n/a
BAR null n/a
---------------- ----------------- ---------------------------------------------

The real environment variables defined in the server always win over the
env vars created by the ``.env`` files.
# you can also filter the list of env vars by name:
$ php bin/console debug:container --env-vars foo

The ``.env`` and ``.env.<environment>`` files should be committed to the shared
repository because they are the same for all developers and machines. However,
the env files ending in ``.local`` (``.env.local`` and ``.env.<environment>.local``)
**should not be committed** because only you will use them. In fact, the
``.gitignore`` file that comes with Symfony prevents them from being committed.
# run this command to show all the details for a specific env var:
$ php bin/console debug:container --env-var=FOO

.. caution::
.. versionadded:: 4.3

Applications created before November 2018 had a slightly different system,
involving a ``.env.dist`` file. For information about upgrading, see:
:doc:`configuration/dot-env-changes`.
The option to debug environment variables was introduced in Symfony 4.3.

.. _configuration-accessing-parameters:

Expand Down
Loading