From 82d2058d2e3c1117cea4a1e19336de97d8a25782 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=A9r=C3=A9my=20Deruss=C3=A9?= Date: Wed, 10 Apr 2019 15:41:43 +0200 Subject: [PATCH 1/3] Add the secret documentation --- best_practices.rst | 8 + configuration/env_var_processors.rst | 53 ++++ configuration/secrets.rst | 391 +++++++++++++++++++++++++++ doctrine/pdo_session_storage.rst | 3 +- 4 files changed, 454 insertions(+), 1 deletion(-) create mode 100644 configuration/secrets.rst diff --git a/best_practices.rst b/best_practices.rst index a249ea74d4c..a996e7d83d0 100644 --- a/best_practices.rst +++ b/best_practices.rst @@ -87,6 +87,14 @@ application behavior. :ref:`Use env vars in your project ` to define these options and create multiple ``.env`` files to :ref:`configure env vars per environment `. +Use Secret for Sensitive Information +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +These are the options used to store sensitive information like passwords, +tokens, api key + +:ref:`Use secrets ` to define these options in an easy and secure way. + Use Parameters for Application Configuration ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/configuration/env_var_processors.rst b/configuration/env_var_processors.rst index 67d9bca422e..934c96199f9 100644 --- a/configuration/env_var_processors.rst +++ b/configuration/env_var_processors.rst @@ -624,6 +624,59 @@ Symfony provides the following env var processors: The ``query_string`` processor was introduced in Symfony 4.3. +``env(secret:FOO)`` + Reads a secret value stored in the app's vault, :ref:`see how to set Secrets`. + + .. code-block:: terminal + + $ php bin/console secrets:set DATABASE_PASSWORD - + + .. configuration-block:: + + .. code-block:: yaml + + # config/packages/database.yaml + doctrine: + dbal: + # by convention the env var names are always uppercase + url: '%env(DATABASE_URL)%' + password: '%env(secret:DATABASE_PASSWORD)%' + + .. code-block:: xml + + + + + + + + + + + + + .. code-block:: php + + // config/packages/doctrine.php + $container->loadFromExtension('doctrine', [ + 'dbal' => [ + // by convention the env var names are always uppercase + 'url' => '%env(DATABASE_URL)%', + 'password' => '%env(secret:DATABASE_PASSWORD)%', + ] + ]); + + + .. versionadded:: 4.4 + + The ``secret`` processor was introduced in Symfony 4.4. + It is also possible to combine any number of processors: .. code-block:: yaml diff --git a/configuration/secrets.rst b/configuration/secrets.rst new file mode 100644 index 00000000000..adccfe62892 --- /dev/null +++ b/configuration/secrets.rst @@ -0,0 +1,391 @@ +.. index:: + single: Secrets + +How to Keep Sensitive Information Secret +======================================== + +.. versionadded:: 4.4 + + The Secrets management was introduced in Symfony 4.4. + +In :doc:`/configuration` and :doc:`/configuration/environment_variables`, you +learned how to manage your application configuration. In this article you'll +learn how to safely configure your application with sensitive information such +as credentials, passwords, tokens, API keys without exposing them. + +.. note:: + + In order to use Symfony's secrets, you will need the sodium PHP extension + bundled in PHP 7.2, but if you use an earlier PHP version, you can + install the `libsodium`_ PHP extension or use the + `paragonie/sodium_compat`_ package. + +.. _secrets-configuration: + +Configuration +------------- + +By Default, secrets are enabled by the Framework. The default behaviors can be +configured: + +.. configuration-block:: + + .. code-block:: yaml + + # config/packages/framework.yaml + framework: + secrets: + #vault_directory: '%kernel.project_dir%/config/secrets/%kernel.environment%' + #local_dotenv_file: '%kernel.project_dir%/.env.local' + #decryption_env_var: 'base64:default::SYMFONY_DECRYPTION_SECRET' + + .. code-block:: xml + + + + + + + + + + .. code-block:: php + + // config/packages/framework.php + $container->loadFromExtension('framework', [ + 'secrets' => [ + // 'vault_directory' => '%kernel.project_dir%/config/secrets/%kernel.environment%', + // 'local_dotenv_file' => '%kernel.project_dir%/.env.local', + // 'decryption_env_var' => 'base64:default::SYMFONY_DECRYPTION_SECRET', + ], + ]); + +.. _secrets-generate-keys: + +Generate Cryptographic Keys +--------------------------- + +In order to encrypt and decrypt **secrets**, symfony needs **cryptographic keys**. +This can be done with the provided command ``secrets:generate-keys``. + +.. code-block:: terminal + + $ APP_ENV=prod php bin/console secrets:generate-keys + +This command generates a new pair of asymetric **cryptographic keys** in +``%kernel.project_dir%/config/secrets/%kernel.environment%``. +The **encryption key** is stored in the ``%kernel.environment%.sodium.encrypt.public`` +file, it is used to encrypt secrets when developers add or update secrets. This +key can be committed. The **decryption key** stored in the +``%kernel.environment%.sodium.decrypt.private`` file, it is used to decrypt +secrets and provide the revealed values to services. The number of people who +have access this key should be as small as possible. + +.. caution:: + + The ``prod.sodium.decrypt.private`` file is sensitive and **should not** be + committed nor publicly shared. Your team developers and Continuous + Integration services don't need that key. If the **decryption key** has + been exposed (ex-employee leaving for instance), you should consider + generating a new one with the command ``secrets:generate-keys --rotate``. + +.. _secrets-set: + +Create or Update Secrets +------------------------ + +You can add new secrets with the command ``secrets:set``. Symfony will ask you, +in a hidden prompt, to enter the value. Symfony will encrypt this value and +store it in a file located by default in the folder +``%kernel.project_dir%/config/secrets/%kernel.environment%``. +This file should be committed along side the other project's files. + +.. code-block:: terminal + + # create a "DATABASE_PASSWORD" secret interactively + $ php bin/console secrets:set DATABASE_PASSWORD + + # create secret for the "prod" environment + $ APP_ENV=prod php bin/console secrets:set DATABASE_PASSWORD + + # provide a file where to read the secret from + $ php bin/console secrets:set APPLICATION_CREDENTIAL ~/Download/key.json + + # or contents passed to STDIN + $ echo -n "$AWS_SECRET_ACCESS_KEY" | php bin/console secrets:set AWS_KEY - + + # or let Symfony generate a random one for you + $ php bin/console secrets:set REMEMBER_ME --random + +If the secret already exists, its value will be overridden by the new one. + +.. tip:: + + The ``--random`` flag will display the generated value in the output. + On Linux, you can use the ``xclip`` command to store it directly in your + clipboard: + + .. code-block:: terminal + + $ php bin/console secrets:set REMEMBER_ME --random | xclip -selection c + +.. _secrets-reference: + +Referencing Secrets in Configuration Files +------------------------------------------ + +You can reference the secrets in any configuration files by prefixing their names +using the **secret** :ref:`environment variable processors `. +Their actual values will be resolved at runtime, so that container compilation +and cache warmup don't need the **decryption key**. + +.. configuration-block:: + + .. code-block:: yaml + + # config/packages/doctrine.yaml + doctrine: + dbal: + password: '%env(secret:DATABASE_PASSWORD)%' + # ... + # ... + + .. code-block:: xml + + + + + + + + + + + + .. code-block:: php + + // config/packages/doctrine.php + $container->loadFromExtension('doctrine', [ + 'dbal' => [ + 'password' => '%env(secret:DATABASE_PASSWORD)%', + ] + ]); + +This configuration requires that all environments uses secrets. Each +environment would have its own **cryptographic keys** and their own encrypted +secrets. + +You can also use parameters to configure different strategies per environnement, +by defining a default plain text secret: + +.. configuration-block:: + + .. code-block:: yaml + + # config/packages/doctrine.yaml + doctrine: + dbal: + password: '%database_password%' + # ... + # ... + + parameters: + database_password: 'not a secret' + + .. code-block:: xml + + + + + + + + + + + not a secret + + + + + .. code-block:: php + + // config/packages/doctrine.php + $container->loadFromExtension('doctrine', [ + 'dbal' => [ + 'password' => '%env(secret:DATABASE_PASSWORD)%', + ] + ]); + $container->setParameter('database_password', 'not a secret'); + +Then overriding it in production environment: + +.. configuration-block:: + + .. code-block:: yaml + + # config/packages/prod/doctrine.yaml + parameters: + database_password: '%env(secret:DATABASE_PASSWORD)%' + + .. code-block:: xml + + + + + + + %env(secret:DATABASE_PASSWORD)% + + + + + .. code-block:: php + + // config/packages/prod/doctrine.php + $container->setParameter('database_password', '%env(secret:DATABASE_PASSWORD)%'); + +.. _secrets-list: + +List Existing Secrets +--------------------- + +Everybody is allowed to list the Secrets' names with the command +``secrets:list``. +If you have the **decryption key** you can also reveal the secrets' values by +passing the option ``--reveal`` to the command: + +.. code-block:: terminal + + $ php bin/console secrets:list --reveal + + ------------------- ------------ + Name Value + ------------------- ------------ + DATABASE_PASSWORD "my secret" + ------------------- ------------ + +Remove Secrets +-------------- + +Symfony provides a convenient command to remove a Secret: + +.. code-block:: terminal + + $ php bin/console secrets:remove DATABASE_PASSWORD + +Rotate Secrets +-------------- + +The ``secrets:generate-keys`` command provides an ``--rotate`` option to +regenerate the **cryptographic keys**. +Symfony will decrypt previous secrets with the old key, generate new +**cryptographic keys** and re-encrypt secrets with the new key. +In order to decrypt previous secrets, the developper must have the +**decryption key**. + +Local secrets +------------- + +It's common for developpers to use their own privates secrets (for +instance a Github token, an Ldap password, or a personnal AWS access key, ...). + +The ``secrets:set`` and ``secrets:remove`` commands provide an ``--local`` +option that stores the secrets in the local ``.env.local`` file like a standard +environment variable suffixed with ``_SECRET``. + +This environment variable will take precedence over the original secret (if +exists). + +.. code-block:: terminal + + $ echo -n "root" | php bin/console secrets:set DATABASE_PASSWORD - + +The ``.env.local`` file will look like: + +.. code-block:: bash + + DATABASE_PASSWORD_SECRET=root + +Listing the secrets will now display the local variable too. + +.. code-block:: terminal + + $ php bin/console secrets:remove DATABASE_PASSWORD + ------------------- ------------- ------------- + Name Value Local Value + ------------------- ------------- ------------- + DATABASE_PASSWORD "my secret" "root" + ------------------- ------------- ------------- + +In addition, Symfony provides the ``secrets:decrypt-to-local``command, it's +decrypts all secrets and stores them in the local vault. Symfony also provides +the ``secrets:encrypt-from-local`` command, it's encrypts all local secrets to +the vault. + +.. _secrets-deploy + +Deploy secrets to production +---------------------------- + +As the **decryption key** is not committed, during development, you'll have to +manually deploy it (once and for all). You have 2 ways to do it. + +1) uploading the file + +The first way, is to copy the **decryption key** file stored in +``%kernel.project_dir%/config/secrets/%kernel.environment%/%kernel.environment%.sodium.decrypt.private`` +on the servers. + +2) Using env variable + +The second way is to set the ``SYMFONY_DECRYPTION_SECRET`` environment variable +with the base64 encoded value of the **encryption key**. + +A fancy way to fetch the value of the key is: + +.. code-block:: terminal + + $ php -r 'echo base64_encode(require "config/secrets/prod/prod.sodium.decrypt.private");' + +.. _`libsodium`: https://pecl.php.net/package/libsodium +.. _ +`sodium_compatparagonie/sodium_compat https://packagist.org/packages/paragonie/sodium_compat + +To improve performance, you can also decrypt all secrets and store them in the +local vault with the command: + +.. code-block:: terminal + + $ php bin/console secrets:decrypt-to-local --force diff --git a/doctrine/pdo_session_storage.rst b/doctrine/pdo_session_storage.rst index 7bc6bacc450..e33486e7718 100644 --- a/doctrine/pdo_session_storage.rst +++ b/doctrine/pdo_session_storage.rst @@ -70,7 +70,8 @@ To use it, first register a new handler service: Configure the database credentials :ref:`using environment variables in the config file ` - to make your application more secure. + or :ref:`using secrets in the config file ` to make + your application more secure. Next, tell Symfony to use your service as the session handler: From 75e5ae65e9d75c78a7de931c71101f81a4a3ad5f Mon Sep 17 00:00:00 2001 From: Ryan Weaver Date: Sat, 18 Jan 2020 15:07:37 -0600 Subject: [PATCH 2/3] finishing the secrets documentation --- best_practices.rst | 6 +- configuration.rst | 214 +++++++------- configuration/env_var_processors.rst | 53 ---- configuration/secrets.rst | 414 +++++++++++---------------- doctrine/pdo_session_storage.rst | 6 +- 5 files changed, 286 insertions(+), 407 deletions(-) diff --git a/best_practices.rst b/best_practices.rst index a996e7d83d0..5616f1f8120 100644 --- a/best_practices.rst +++ b/best_practices.rst @@ -90,10 +90,8 @@ and create multiple ``.env`` files to :ref:`configure env vars per environment < Use Secret for Sensitive Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -These are the options used to store sensitive information like passwords, -tokens, api key - -:ref:`Use secrets ` to define these options in an easy and secure way. +When your application has sensitive configuration - like an API key - you should +store those securely via :doc:`secrets `. Use Parameters for Application Configuration ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/configuration.rst b/configuration.rst index 8fe6169b8ec..caa2a25cb61 100644 --- a/configuration.rst +++ b/configuration.rst @@ -436,14 +436,14 @@ Configuration Based on Environment Variables Using `environment variables`_ (or "env vars" for short) is a common practice to configure options that depend on where the application is run (e.g. the database -credentials are usually different in production and in your local machine). +credentials are usually different in production versus your local machine). If +the values are sensitive, you can even :doc:`encrypt them as secrets `. -Instead of defining those as regular options, you can define them as environment -variables and reference them in the configuration files using the special syntax +You can reference environment variables using the special syntax ``%env(ENV_VAR_NAME)%``. The values of these options are resolved at runtime (only once per request, to not impact performance). -This example shows how to configure the database connection using an env var: +This example shows how you could configure the database connection using an env var: .. configuration-block:: @@ -485,164 +485,168 @@ This example shows how to configure the database connection using an env var: ] ]); -The next step is to define the value of those env vars in your shell, your web -server, etc. This is explained in the following sections, but to protect your -application from undefined env vars, you can give them a default value using the -``.env`` file: - -.. code-block:: bash - - # .env - DATABASE_URL=sqlite:///%kernel.project_dir%/var/data.db - .. seealso:: The values of env vars can only be strings, but Symfony includes some :doc:`env var processors ` to transform their contents (e.g. to turn a string value into an integer). -In order to define the actual values of env vars, Symfony proposes different -solutions depending if the application is running in production or in your local -development machine. - -Independent from the way you set environment variables, you may need to run the -``debug:container`` command with the ``--env-vars`` option to verify that they -are defined and have the expected values: +To define the value of an env var, you have several options: -.. code-block:: terminal +* :ref:`Add the value to a .env file `; +* :ref:`Encrypt the value as a secret `; +* Set the value as a real environment variable in your shell or your web server. - $ php bin/console debug:container --env-vars - - ---------------- ----------------- --------------------------------------------- - Name Default value Real value - ---------------- ----------------- --------------------------------------------- - APP_SECRET n/a "471a62e2d601a8952deb186e44186cb3" - FOO "[1, "2.5", 3]" n/a - BAR null n/a - ---------------- ----------------- --------------------------------------------- +.. tip:: - # you can also filter the list of env vars by name: - $ php bin/console debug:container --env-vars foo + Some hosts - like SymfonyCloud - offer easy `utilities to manage env vars`_ + in production. - # run this command to show all the details for a specific env var: - $ php bin/console debug:container --env-var=FOO +.. caution:: -.. versionadded:: 4.3 + Beware that dumping the contents of the ``$_SERVER`` and ``$_ENV`` variables + or outputting the ``phpinfo()`` contents will display the values of the + environment variables, exposing sensitive information such as the database + credentials. - The option to debug environment variables was introduced in Symfony 4.3. + The values of the env vars are also exposed in the web interface of the + :doc:`Symfony profiler `. In practice this shouldn't be a + problem because the web profiler must **never** be enabled in production. .. _configuration-env-var-in-dev: .. _config-dot-env: -Configuring Environment Variables in Development -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Configuring Environment Variables in .env Files +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Instead of defining env vars in your shell or your web server, Symfony proposes -a convenient way of defining them in your local machine based on a file called -``.env`` (with a leading dot) located at the root of your project. +Instead of defining env vars in your shell or your web server, Symfony provides +a convenient way to define them inside a ``.env`` (with a leading dot) file +located at the root of your project. The ``.env`` file is read and parsed on every request and its env vars are added -to the ``$_ENV`` PHP variable. The existing env vars are never overwritten by -the values defined in ``.env``, so you can combine both. +to the ``$_ENV`` & ``$_SERVER`` PHP variables. Any existing env vars are *never* +overwritten by the values defined in ``.env``, so you can combine both. -This is for example the content of the ``.env`` file to define the value of the -``DATABASE_URL`` env var shown earlier in this article: +For example, to define the ``DATABASE_URL`` env var shown earlier in this article, +you can add: .. code-block:: bash # .env DATABASE_URL="mysql://db_user:db_password@127.0.0.1:3306/db_name" +This file should be committed to your repository and (due to that fact) should +only contain "default" values that are good for local development. This file +should not contain production values. + In addition to your own env vars, this ``.env`` file also contains the env vars defined by the third-party packages installed in your application (they are added automatically by :ref:`Symfony Flex ` when installing packages). +.. _configuration-multiple-env-files: + +Overriding Environment Values via .env.local +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you need to override an environment value (e.g. to a different value on your +local machine), you can do that in a ``.env.local`` file: + +.. code-block:: bash + + # .env.local + DATABASE_URL="mysql://root:@127.0.0.1:3306/my_database_name" + +This file should be ignored by git and should *not* be committed to your repository. +Several other ``.env`` files are available to set environment variables in *just* +the right situation: + +* ``.env``: defines the default values of the env vars needed by the application; +* ``.env.local``: overrides the default values for all environments but only on + the machine which contains the file. This file should not be committed to the + repository and it's ignored in the ``test`` environment (because tests should + produce the same results for everyone); +* ``.env.`` (e.g. ``.env.test``): overrides env vars only for one + environment but for all machines (these files *are* committed); +* ``.env..local`` (e.g. ``.env.test.local``): defines machine-specific + env var overrides only for one environment. It's similar to ``.env.local``, + but the overrides only apply to one environment. + +*Real* environment variables always win over env vars created by any of the +``.env`` files. + +The ``.env`` and ``.env.`` files should be committed to the +repository because they are the same for all developers and machines. However, +the env files ending in ``.local`` (``.env.local`` and ``.env..local``) +**should not be committed** because only you will use them. In fact, the +``.gitignore`` file that comes with Symfony prevents them from being committed. + +.. caution:: + + Applications created before November 2018 had a slightly different system, + involving a ``.env.dist`` file. For information about upgrading, see: + :doc:`configuration/dot-env-changes`. + .. _configuration-env-var-in-prod: Configuring Environment Variables in Production ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -In production, the ``.env`` files are also parsed and loaded on each request so -you can add env vars to those already defined in the server. In order to improve -performance, you can run the ``dump-env`` command (available when using -:ref:`Symfony Flex ` 1.2 or later). +In production, the ``.env`` files are also parsed and loaded on each request. So +the easiest way to define env vars is by deploying a ``.env.local`` file to your +production server(s) with your production values. -This command parses all the ``.env`` files once and compiles their contents into -a new PHP-optimized file called ``.env.local.php``. From that moment, Symfony -will load the parsed file instead of parsing the ``.env`` files again: +To improve performance, you can optionally run the ``dump-env`` command (available +in :ref:`Symfony Flex ` 1.2 or later): .. code-block:: terminal + # parses ALL .env files and dumps their final values to .env.local.php $ composer dump-env prod +After running this command, Symfony will load the ``.env.local.php`` file to +get the environment variables and will not spend time parsing the ``.env`` files. + .. tip:: Update your deployment tools/workflow to run the ``dump-env`` command after each deploy to improve the application performance. -.. _configuration-env-var-web-server: +.. _configuration-secrets: -Creating ``.env`` files is the easiest way of using env vars in Symfony -applications. However, you can also configure real env vars in your servers and -operating systems. +Encrypting Environment Variables (Secrets) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. tip:: - - SymfonyCloud, the cloud service optimized for Symfony applications, defines - some `utilities to manage env vars`_ in production. - -.. caution:: +Instead of defining a real environment variable or adding it to a ``.env`` file, +if the value of a variable is sensitive (e.g. an API key or a database password), +you can encrypt the value using the :doc:`secrets management system `. - Beware that dumping the contents of the ``$_SERVER`` and ``$_ENV`` variables - or outputting the ``phpinfo()`` contents will display the values of the - environment variables, exposing sensitive information such as the database - credentials. - - The values of the env vars are also exposed in the web interface of the - :doc:`Symfony profiler `. In practice this shouldn't be a - problem because the web profiler must **never** be enabled in production. - -.. _configuration-multiple-env-files: - -Managing Multiple .env Files -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Listing Environment Variables +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The ``.env`` file defines the default values for all env vars. However, it's -common to override some of those values depending on the environment (e.g. to -use a different database for tests) or depending on the machine (e.g. to use a -different OAuth token on your local machine while developing). +Regardless of how you set environment variables, you can see a full list with +their values by running: -That's why you can define multiple ``.env`` files to override env vars. The -following list shows the files loaded in all environments. The ``.env`` file is -the only mandatory file and each file content overrides the previous one: +.. code-block:: terminal -* ``.env``: defines the default values of the env vars needed by the application; -* ``.env.local``: overrides the default values of env vars for all environments - but only in the machine which contains the file (e.g. your development computer). - This file should not be committed to the repository and it's ignored in the - ``test`` environment (because tests should produce the same results for everyone); -* ``.env.`` (e.g. ``.env.test``): overrides env vars only for some - environment but for all machines; -* ``.env..local`` (e.g. ``.env.test.local``): defines machine-specific - env vars overrides only for some environment. It's similar to ``.env.local``, - but the overrides only apply to some particular environment. + $ php bin/console debug:container --env-vars -.. note:: + ---------------- ----------------- --------------------------------------------- + Name Default value Real value + ---------------- ----------------- --------------------------------------------- + APP_SECRET n/a "471a62e2d601a8952deb186e44186cb3" + FOO "[1, "2.5", 3]" n/a + BAR null n/a + ---------------- ----------------- --------------------------------------------- - The real environment variables defined in the server always win over the - env vars created by the ``.env`` files. + # you can also filter the list of env vars by name: + $ php bin/console debug:container --env-vars foo -The ``.env`` and ``.env.`` files should be committed to the shared -repository because they are the same for all developers and machines. However, -the env files ending in ``.local`` (``.env.local`` and ``.env..local``) -**should not be committed** because only you will use them. In fact, the -``.gitignore`` file that comes with Symfony prevents them from being committed. + # run this command to show all the details for a specific env var: + $ php bin/console debug:container --env-var=FOO -.. caution:: +.. versionadded:: 4.3 - Applications created before November 2018 had a slightly different system, - involving a ``.env.dist`` file. For information about upgrading, see: - :doc:`configuration/dot-env-changes`. + The option to debug environment variables was introduced in Symfony 4.3. .. _configuration-accessing-parameters: diff --git a/configuration/env_var_processors.rst b/configuration/env_var_processors.rst index 934c96199f9..67d9bca422e 100644 --- a/configuration/env_var_processors.rst +++ b/configuration/env_var_processors.rst @@ -624,59 +624,6 @@ Symfony provides the following env var processors: The ``query_string`` processor was introduced in Symfony 4.3. -``env(secret:FOO)`` - Reads a secret value stored in the app's vault, :ref:`see how to set Secrets`. - - .. code-block:: terminal - - $ php bin/console secrets:set DATABASE_PASSWORD - - - .. configuration-block:: - - .. code-block:: yaml - - # config/packages/database.yaml - doctrine: - dbal: - # by convention the env var names are always uppercase - url: '%env(DATABASE_URL)%' - password: '%env(secret:DATABASE_PASSWORD)%' - - .. code-block:: xml - - - - - - - - - - - - - .. code-block:: php - - // config/packages/doctrine.php - $container->loadFromExtension('doctrine', [ - 'dbal' => [ - // by convention the env var names are always uppercase - 'url' => '%env(DATABASE_URL)%', - 'password' => '%env(secret:DATABASE_PASSWORD)%', - ] - ]); - - - .. versionadded:: 4.4 - - The ``secret`` processor was introduced in Symfony 4.4. - It is also possible to combine any number of processors: .. code-block:: yaml diff --git a/configuration/secrets.rst b/configuration/secrets.rst index adccfe62892..edd4d470d84 100644 --- a/configuration/secrets.rst +++ b/configuration/secrets.rst @@ -8,192 +8,105 @@ How to Keep Sensitive Information Secret The Secrets management was introduced in Symfony 4.4. -In :doc:`/configuration` and :doc:`/configuration/environment_variables`, you -learned how to manage your application configuration. In this article you'll -learn how to safely configure your application with sensitive information such -as credentials, passwords, tokens, API keys without exposing them. +:ref:`Environment variables ` are the best way to store configuration +that depends on where the application is run - for example, some API key that +might be set to one value while developing locally and another value on production. + +When these values are *sensitive* and need to be kept private, you can safely +store them by using Symfony's secrets management system - sometimes called a +"vault". .. note:: - In order to use Symfony's secrets, you will need the sodium PHP extension - bundled in PHP 7.2, but if you use an earlier PHP version, you can + The Secrets system requires the sodium PHP extension that is bundled + with PHP 7.2. If you're using an earlier PHP version, you can install the `libsodium`_ PHP extension or use the `paragonie/sodium_compat`_ package. -.. _secrets-configuration: - -Configuration -------------- - -By Default, secrets are enabled by the Framework. The default behaviors can be -configured: - -.. configuration-block:: - - .. code-block:: yaml +.. _secrets-generate-keys: - # config/packages/framework.yaml - framework: - secrets: - #vault_directory: '%kernel.project_dir%/config/secrets/%kernel.environment%' - #local_dotenv_file: '%kernel.project_dir%/.env.local' - #decryption_env_var: 'base64:default::SYMFONY_DECRYPTION_SECRET' +Generate Cryptographic Keys +--------------------------- - .. code-block:: xml +In order to encrypt and decrypt **secrets**, symfony needs **cryptographic keys**. +A pair of keys can be generated by running: - - - - - - - +.. code-block:: terminal - .. code-block:: php + $ php bin/console secrets:generate-keys - // config/packages/framework.php - $container->loadFromExtension('framework', [ - 'secrets' => [ - // 'vault_directory' => '%kernel.project_dir%/config/secrets/%kernel.environment%', - // 'local_dotenv_file' => '%kernel.project_dir%/.env.local', - // 'decryption_env_var' => 'base64:default::SYMFONY_DECRYPTION_SECRET', - ], - ]); +This will generate a pair of asymmetric **cryptographic keys**. Each +:ref:`environment ` has its own set of keys. Assuming +you're coding locally in the ``dev`` environment, this will create: -.. _secrets-generate-keys: +``config/secrets/dev/dev.encrypt.public.php`` + Used to encrypt/add secrets to the vault. Can be safely committed. -Generate Cryptographic Keys ---------------------------- +``config/secrets/dev/dev.decrypt.private.php`` + Used to decrypt/read secrets from the vault. The ``dev`` decryption key can + be committed (assuming no highly-sensitive secrets are stored in the dev vault) + but the ``prod`` decryption key should *never* be committed. -In order to encrypt and decrypt **secrets**, symfony needs **cryptographic keys**. -This can be done with the provided command ``secrets:generate-keys``. +You can generate a pair of cryptographic keys for the ``prod`` environment by +running: .. code-block:: terminal - $ APP_ENV=prod php bin/console secrets:generate-keys + $ php bin/console secrets:generate-keys --env=prod -This command generates a new pair of asymetric **cryptographic keys** in -``%kernel.project_dir%/config/secrets/%kernel.environment%``. -The **encryption key** is stored in the ``%kernel.environment%.sodium.encrypt.public`` -file, it is used to encrypt secrets when developers add or update secrets. This -key can be committed. The **decryption key** stored in the -``%kernel.environment%.sodium.decrypt.private`` file, it is used to decrypt -secrets and provide the revealed values to services. The number of people who -have access this key should be as small as possible. +This will generate ``config/secrets/prod/prod.encrypt.public.php`` and +``config/secrets/prod/prod.decrypt.private.php``. .. caution:: - The ``prod.sodium.decrypt.private`` file is sensitive and **should not** be - committed nor publicly shared. Your team developers and Continuous - Integration services don't need that key. If the **decryption key** has - been exposed (ex-employee leaving for instance), you should consider - generating a new one with the command ``secrets:generate-keys --rotate``. + The ``prod.decrypt.private.php`` file is highly sensitive. Your team of developers + and even Continuous Integration services don't need that key. If the + **decryption key** has been exposed (ex-employee leaving for instance), you + should consider generating a new one by running: + ``secrets:generate-keys --rotate``. .. _secrets-set: Create or Update Secrets ------------------------ -You can add new secrets with the command ``secrets:set``. Symfony will ask you, -in a hidden prompt, to enter the value. Symfony will encrypt this value and -store it in a file located by default in the folder -``%kernel.project_dir%/config/secrets/%kernel.environment%``. -This file should be committed along side the other project's files. +Suppose you want to store your database password a secret. By using the +``secrets:set`` command, you should add this secret to both the ``dev`` *and* +``prod`` vaults: .. code-block:: terminal - # create a "DATABASE_PASSWORD" secret interactively + # the input is hidden as you type for security + + # set your a default development value (can be overridden locally) $ php bin/console secrets:set DATABASE_PASSWORD - # create secret for the "prod" environment - $ APP_ENV=prod php bin/console secrets:set DATABASE_PASSWORD + # set your production value + $ php bin/console secrets:set DATABASE_PASSWORD --env=prod + +This will create a new file for the secret in ``config/secrets/dev`` and another +in ``config/secrets/prod``. You can also set the secret in a few other ways: + +.. code-block:: terminal # provide a file where to read the secret from - $ php bin/console secrets:set APPLICATION_CREDENTIAL ~/Download/key.json + $ php bin/console secrets:set DATABASE_PASSWORD ~/Download/password.json # or contents passed to STDIN - $ echo -n "$AWS_SECRET_ACCESS_KEY" | php bin/console secrets:set AWS_KEY - + $ echo -n "$DB_PASS" | php bin/console secrets:set DATABASE_PASSWORD - - # or let Symfony generate a random one for you + # or let Symfony generate a random value for you $ php bin/console secrets:set REMEMBER_ME --random -If the secret already exists, its value will be overridden by the new one. - -.. tip:: - - The ``--random`` flag will display the generated value in the output. - On Linux, you can use the ``xclip`` command to store it directly in your - clipboard: - - .. code-block:: terminal - - $ php bin/console secrets:set REMEMBER_ME --random | xclip -selection c - -.. _secrets-reference: - Referencing Secrets in Configuration Files ------------------------------------------ -You can reference the secrets in any configuration files by prefixing their names -using the **secret** :ref:`environment variable processors `. -Their actual values will be resolved at runtime, so that container compilation -and cache warmup don't need the **decryption key**. - -.. configuration-block:: - - .. code-block:: yaml - - # config/packages/doctrine.yaml - doctrine: - dbal: - password: '%env(secret:DATABASE_PASSWORD)%' - # ... - # ... - - .. code-block:: xml - - - - +Secret values can be referenced in the same way as +:ref:`environment variables`. Be careful that you don't +accidentally define a secret *and* an environment variable with the same name: +**environment variables override secrets**. - - - - - - - .. code-block:: php - - // config/packages/doctrine.php - $container->loadFromExtension('doctrine', [ - 'dbal' => [ - 'password' => '%env(secret:DATABASE_PASSWORD)%', - ] - ]); - -This configuration requires that all environments uses secrets. Each -environment would have its own **cryptographic keys** and their own encrypted -secrets. - -You can also use parameters to configure different strategies per environnement, -by defining a default plain text secret: +If you stored a ``DATABASE_PASSWORD`` secret, you can reference by: .. configuration-block:: @@ -202,13 +115,10 @@ by defining a default plain text secret: # config/packages/doctrine.yaml doctrine: dbal: - password: '%database_password%' + password: '%env(DATABASE_PASSWORD)%' # ... # ... - parameters: - database_password: 'not a secret' - .. code-block:: xml @@ -227,10 +137,6 @@ by defining a default plain text secret: /> - - not a secret - - .. code-block:: php @@ -241,60 +147,26 @@ by defining a default plain text secret: 'password' => '%env(secret:DATABASE_PASSWORD)%', ] ]); - $container->setParameter('database_password', 'not a secret'); - -Then overriding it in production environment: - -.. configuration-block:: - - .. code-block:: yaml - - # config/packages/prod/doctrine.yaml - parameters: - database_password: '%env(secret:DATABASE_PASSWORD)%' - - .. code-block:: xml - - - - - - %env(secret:DATABASE_PASSWORD)% - - - - - .. code-block:: php - - // config/packages/prod/doctrine.php - $container->setParameter('database_password', '%env(secret:DATABASE_PASSWORD)%'); - -.. _secrets-list: +The actual value will be resolved at runtime: container compilation and cache +warmup don't need the **decryption key**. List Existing Secrets --------------------- -Everybody is allowed to list the Secrets' names with the command -``secrets:list``. -If you have the **decryption key** you can also reveal the secrets' values by -passing the option ``--reveal`` to the command: +Everybody is allowed to list the secrets names with the command +``secrets:list``. If you have the **decryption key** you can also reveal the +secrets' values by passing the ``--reveal`` option: .. code-block:: terminal $ php bin/console secrets:list --reveal - ------------------- ------------ - Name Value - ------------------- ------------ + ------------------- ------------ ------------- + Name Value Local Value + ------------------- ------------ ------------- DATABASE_PASSWORD "my secret" - ------------------- ------------ + ------------------- ------------ ------------- Remove Secrets -------------- @@ -305,87 +177,145 @@ Symfony provides a convenient command to remove a Secret: $ php bin/console secrets:remove DATABASE_PASSWORD -Rotate Secrets --------------- - -The ``secrets:generate-keys`` command provides an ``--rotate`` option to -regenerate the **cryptographic keys**. -Symfony will decrypt previous secrets with the old key, generate new -**cryptographic keys** and re-encrypt secrets with the new key. -In order to decrypt previous secrets, the developper must have the -**decryption key**. - -Local secrets -------------- - -It's common for developpers to use their own privates secrets (for -instance a Github token, an Ldap password, or a personnal AWS access key, ...). +Local secrets: Overriding Secrets Locally +----------------------------------------- -The ``secrets:set`` and ``secrets:remove`` commands provide an ``--local`` -option that stores the secrets in the local ``.env.local`` file like a standard -environment variable suffixed with ``_SECRET``. +The ``dev`` environment secrets should contain nice default values for development. +But sometimes a developer *still* needs to override a secret value locally when +developing. -This environment variable will take precedence over the original secret (if -exists). +Most of the ``secrets`` commands - including ``secrets:set`` - have a ``--local`` +option that stores the "secret" in the ``.env.{env}.local`` file as a standard +environment variable. To override the ``DATABASE_PASSWORD`` secret locally, run: .. code-block:: terminal - $ echo -n "root" | php bin/console secrets:set DATABASE_PASSWORD - + $ php bin/console secrets:set DATABASE_PASSWORD --local -The ``.env.local`` file will look like: +If you entered ``root``, you will now see this in your ``.env.dev.local`` file: .. code-block:: bash - DATABASE_PASSWORD_SECRET=root + DATABASE_PASSWORD=root -Listing the secrets will now display the local variable too. +This will *override* the ``DATABASE_PASSWORD`` secret because environment variables +always "win" over secrets. + +Listing the secrets will now also display the local variable: .. code-block:: terminal - $ php bin/console secrets:remove DATABASE_PASSWORD + $ php bin/console secrets:list --reveal ------------------- ------------- ------------- Name Value Local Value ------------------- ------------- ------------- - DATABASE_PASSWORD "my secret" "root" + DATABASE_PASSWORD "dev value" "root" ------------------- ------------- ------------- -In addition, Symfony provides the ``secrets:decrypt-to-local``command, it's -decrypts all secrets and stores them in the local vault. Symfony also provides -the ``secrets:encrypt-from-local`` command, it's encrypts all local secrets to -the vault. +Symfony also provides the ``secrets:decrypt-to-local`` command to decrypts +all secrets and stores them in the local vault and ``secrets:encrypt-from-local`` +to encrypt all local secrets to the vault. -.. _secrets-deploy +Secrets in the test Environment +------------------------------- + +If you add a secret in the ``dev`` and ``prod`` environments, it will be missing +from the ``test`` environment. You *could* create a "vault" for the ``test`` +environment and define the secrets there. But an easier way is to set the test +values via the ``.env.test`` file: + +.. code-block:: bash + + # .env.test + DATABASE_PASSWORD="testing" Deploy secrets to production ---------------------------- -As the **decryption key** is not committed, during development, you'll have to -manually deploy it (once and for all). You have 2 ways to do it. +Because the **decryption key** is not committed, you'll need to manually +store this file somewhere and deploy it. There are 2 ways to do that: -1) uploading the file +1) Uploading the file: -The first way, is to copy the **decryption key** file stored in -``%kernel.project_dir%/config/secrets/%kernel.environment%/%kernel.environment%.sodium.decrypt.private`` -on the servers. +The first option is to copy the **decryption key** - +``/config/secrets/prod/prod.decrypt.private.php`` to your server(s). -2) Using env variable +2) Using an Environment Variable The second way is to set the ``SYMFONY_DECRYPTION_SECRET`` environment variable -with the base64 encoded value of the **encryption key**. +to the base64 encoded value of the **decryption key**. A fancy way to fetch the +value of the key is: + +.. code-block:: terminal -A fancy way to fetch the value of the key is: + $ php -r 'echo base64_encode(require "config/secrets/prod/prod.decrypt.private.php");' + +To improve performance (i.e. avoid decrypting secrets at runtime), you can decrypt +your secrets during deployment to the "local" vault: .. code-block:: terminal - $ php -r 'echo base64_encode(require "config/secrets/prod/prod.sodium.decrypt.private");' + $ php bin/console secrets:decrypt-to-local --force --env=prod -.. _`libsodium`: https://pecl.php.net/package/libsodium -.. _ -`sodium_compatparagonie/sodium_compat https://packagist.org/packages/paragonie/sodium_compat +This will put all the decrypted secrets into ``.env.php.local``. After doing this, +the decryption key does *not* need to remain on the server. -To improve performance, you can also decrypt all secrets and store them in the -local vault with the command: +Rotating Secrets +---------------- -.. code-block:: terminal +The ``secrets:generate-keys`` command provides a ``--rotate`` option to +regenerate the **cryptographic keys**. Symfony will decrypt existing secrets with +the old key, generate new **cryptographic keys** and re-encrypt secrets with the +new key. In order to decrypt previous secrets, the developer must have the +**decryption key**. + +Configuration +------------- + +The secrets system is enabled by default and some of its behavior can be configured: + +.. configuration-block:: + + .. code-block:: yaml + + # config/packages/framework.yaml + framework: + secrets: + #vault_directory: '%kernel.project_dir%/config/secrets/%kernel.environment%' + #local_dotenv_file: '%kernel.project_dir%/.env.%kernel.environment%.local' + #decryption_env_var: 'base64:default::SYMFONY_DECRYPTION_SECRET' + + .. code-block:: xml + + + + + + + + - $ php bin/console secrets:decrypt-to-local --force + .. code-block:: php + + // config/packages/framework.php + $container->loadFromExtension('framework', [ + 'secrets' => [ + // 'vault_directory' => '%kernel.project_dir%/config/secrets/%kernel.environment%', + // 'local_dotenv_file' => '%kernel.project_dir%/.env.%kernel.environment%.local', + // 'decryption_env_var' => 'base64:default::SYMFONY_DECRYPTION_SECRET', + ], + ]); + + +.. _`libsodium`: https://pecl.php.net/package/libsodium +.. _`sodium_compatparagonie/sodium_compat`: https://packagist.org/packages/paragonie/sodium_compat +.. _`paragonie/sodium_compat`: https://github.com/paragonie/sodium_compat diff --git a/doctrine/pdo_session_storage.rst b/doctrine/pdo_session_storage.rst index e33486e7718..8123a68460d 100644 --- a/doctrine/pdo_session_storage.rst +++ b/doctrine/pdo_session_storage.rst @@ -69,9 +69,9 @@ To use it, first register a new handler service: .. tip:: Configure the database credentials - :ref:`using environment variables in the config file ` - or :ref:`using secrets in the config file ` to make - your application more secure. + :ref:`using environment variables ` + or as a :doc:`secret ` to make your application + more secure. Next, tell Symfony to use your service as the session handler: From cd066cc0b7522ee1b2be6528e8b2be7dc5cc9044 Mon Sep 17 00:00:00 2001 From: Ryan Weaver Date: Sun, 26 Jan 2020 12:34:18 -0500 Subject: [PATCH 3/3] tweaks thanks to review --- configuration/secrets.rst | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/configuration/secrets.rst b/configuration/secrets.rst index edd4d470d84..f1294143732 100644 --- a/configuration/secrets.rst +++ b/configuration/secrets.rst @@ -28,7 +28,7 @@ store them by using Symfony's secrets management system - sometimes called a Generate Cryptographic Keys --------------------------- -In order to encrypt and decrypt **secrets**, symfony needs **cryptographic keys**. +In order to encrypt and decrypt **secrets**, Symfony needs **cryptographic keys**. A pair of keys can be generated by running: .. code-block:: terminal @@ -199,7 +199,7 @@ If you entered ``root``, you will now see this in your ``.env.dev.local`` file: DATABASE_PASSWORD=root This will *override* the ``DATABASE_PASSWORD`` secret because environment variables -always "win" over secrets. +always take precedence over secrets. Listing the secrets will now also display the local variable: @@ -229,11 +229,11 @@ values via the ``.env.test`` file: # .env.test DATABASE_PASSWORD="testing" -Deploy secrets to production +Deploy Secrets to Production ---------------------------- -Because the **decryption key** is not committed, you'll need to manually -store this file somewhere and deploy it. There are 2 ways to do that: +Due to the fact that decryption keys should never be committed, you will need to +manually store this file somewhere and deploy it. There are 2 ways to do that: 1) Uploading the file: @@ -317,5 +317,4 @@ The secrets system is enabled by default and some of its behavior can be configu .. _`libsodium`: https://pecl.php.net/package/libsodium -.. _`sodium_compatparagonie/sodium_compat`: https://packagist.org/packages/paragonie/sodium_compat .. _`paragonie/sodium_compat`: https://github.com/paragonie/sodium_compat