Skip to content

expanding on credential usage docs #505

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Nov 14, 2019
Merged

expanding on credential usage docs #505

Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
bfe90fd
expanding on credential usage docs to explicitly call out how to use …
filmaj Nov 12, 2019
e894f75
Grammar fixup.
dobooth Nov 12, 2019
6dd9655
remove vault-based credentials usage details for now
filmaj Nov 13, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 22 additions & 14 deletions docs/credentials.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Credentials

When you test functionality that involves external services such as UPS, FedEx, PayPal, or SignifyD,
When you test functionality that involves external services such as UPS, FedEx, PayPal, or SignifyD,
use the MFTF credentials feature to hide sensitive [data][] like integration tokens and API keys.

Currently the MFTF supports two types of credential storage:
Expand Down Expand Up @@ -53,9 +53,9 @@ magento/carriers_usps_password=Lmgxvrq89uPwECeV
#magento/carriers_dhl_id_us=dhl_test_user
#magento/carriers_dhl_password_us=Mlgxv3dsagVeG
....
```
```

Or add new key & value pairs for your own credentials. The keys use the following format:
Or add new key/value pairs for your own credentials. The keys use the following format:

```conf
<vendor>/<key_name>=<key_value>
Expand All @@ -64,7 +64,7 @@ Or add new key & value pairs for your own credentials. The keys use the followin
<div class="bs-callout bs-callout-info" markdown="1">
The `/` symbol is not supported in a `key_name` other than the one after your vendor or extension name.
</div>

Otherwise you are free to use any other `key_name` you like, as they are merely the keys to reference from your tests.

```conf
Expand All @@ -74,10 +74,10 @@ vendor/my_awesome_service_token=rRVSVnh3cbDsVG39oTMz4A

## Configure Vault Storage

Hashicorp vault secures, stores, and tightly controls access to data in modern computing.
It provides advanced data protection for your testing credentials.
Hashicorp vault secures, stores, and tightly controls access to data in modern computing.
It provides advanced data protection for your testing credentials.

The MFTF works with both `vault enterprise` and `vault open source` that use `KV Version 2` secret engine.
The MFTF works with both `vault enterprise` and `vault open source` that use `KV Version 2` secret engine.

### Install vault CLI

Expand All @@ -95,8 +95,8 @@ vault login -method -path

### Store secrets in vault

The MFTF uses the `KV Version 2` secret engine for secret storage.
More information for working with `KV Version 2` can be found in [Vault KV2][Vault KV2].
The MFTF uses the `KV Version 2` secret engine for secret storage.
More information for working with `KV Version 2` can be found in [Vault KV2][Vault KV2].

#### Secrets path and key convention

Expand Down Expand Up @@ -125,9 +125,9 @@ vault kv put secret/mftf/magento/carriers_usps_password carriers_usps_password=L

### Setup MFTF to use vault

Add vault configuration environment variables [`CREDENTIAL_VAULT_ADDRESS`][] and [`CREDENTIAL_VAULT_SECRET_BASE_PATH`][]
Add vault configuration environment variables [`CREDENTIAL_VAULT_ADDRESS`][] and [`CREDENTIAL_VAULT_SECRET_BASE_PATH`][]
from `etc/config/.env.example` in `.env`.
Set values according to your vault server configuration.
Set values according to your vault server configuration.

```conf
# Default vault dev server
Expand All @@ -137,7 +137,7 @@ CREDENTIAL_VAULT_SECRET_BASE_PATH=secret

## Configure both File Storage and Vault Storage

It is possible and sometimes useful to setup and use both `.credentials` file and vault for secret storage at the same time.
It is possible and sometimes useful to setup and use both `.credentials` file and vault for secret storage at the same time.
In this case, the MFTF tests are able to read secret data at runtime from both storage options, but the local `.credentials` file will take precedence.

<!-- {% raw %} -->
Expand All @@ -150,11 +150,19 @@ Define the value as a reference to the corresponding key in the credentials file

- `_CREDS` is an environment constant pointing to the `.credentials` file
- `my_data_key` is a key in the the `.credentials` file or vault that contains the value to be used in a test step
- for File Storage, ensure your key contains the vendor prefix, i.e. `vendor/my_data_key`
- for Vault Storage, ensure your key contains your secret basepath + `/mftf/` + your vendor prefix + `/` and finally your key, i.e. `secret/mftf/vendor/my_data_key`

For example, to reference secret data in the [`fillField`][] action, use the `userInput` attribute using a typical File Storage:

```xml
<fillField stepKey="FillApiToken" selector=".api-token" userInput="{{_CREDS.vendor/my_data_key}}" />
```

For example, reference secret data in the [`fillField`][] action with the `userInput` attribute.
To do the same using Vault Storage:

```xml
<fillField stepKey="FillApiToken" selector=".api-token" userInput="{{_CREDS.my_data_key}}" />
<fillField stepKey="FillApiToken" selector=".api-token" userInput="{{_CREDS.secret/mftf/vendor/my_data_key}}" />
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually incorrect, you need to be able to just use vendor/my_data_key for both (MFTF checks file storage, then vault).

If you're having issues with getting MFTF to read the vault version, see this section of devdocs:
https://devdocs.magento.com/mftf/docs/credentials.html#setup-mftf-to-use-vault

I think you just need to set this in .env:
CREDENTIAL_VAULT_SECRET_BASE_PATH=secret
MFTF should automatically call <CREDENTIAL_VAULT_ADDRESS>/<CREDENTIAL_VAULT_SECRET_BASE_PATH>/mftf/<vendor/my_data_key> in your case.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @KevinBKozan ! I actually do not have Vault set up locally, so I was completely shooting in the dark with the Vault portion of the docs.

Do you have suggestions on how to modify this section? Ideally, I would imagine the best developer experience here would be to use the same token to access the secret regardless of the underlying storage leveraged. If that is the case, then I can definitely simplify the wording in the docs here to make that clear! Just let me know.

Copy link
Contributor

@KevinBKozan KevinBKozan Nov 13, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha. I think some background info is kind of necessary to be able to make good changes to the docs, but the tl;dr is Vault integration with MFTF is about 50% done in the whole picture (framework integration is complete, build/EQP integration is not).

The primary purpose of the vault integration is actually to connect to the (as of yet, non-existant) MFTF vault credential store. This is the thing that builds are going to be using to run tests, so it becomes the "canonical" list of credentials that if you submit an extension, you can submit creds to your sandbox or testing environment.

So with that said, for local test running the prefered way would actually be for you to just define creds in .credentials as you need them. If you're running a test that requires _CREDS.some_api_key it'd be then on you to define it in .credentials since you would conceivably know the value (if you have access to it).

You can still use vault to resolve these _CREDS references though. So say a test is using _CREDS.some_api_key, the user flow would have to be

  • brew install vault
  • vault login <username/token/whatever EQP decides to use>
    • vault is now authenticated on the machine, MFTF will use this auth to access creds
    • you don't need to tell vault where the server is AFAIK, it will know by your token to map it to the right server
  • define details in .env:
    • CREDENTIAL_VAULT_ADDRESS=<whatever it ends up being>
    • CREDENTIAL_VAULT_SECRET_BASE_PATH=<probably just "secret">
    • MFTF now knows what the vault address is, what the base path is, and has auth to access it

Copy link
Contributor

@KevinBKozan KevinBKozan Nov 13, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I guess the followup question is, what are you trying to use vault for, connecting to that central service or a local instance? That may help guide how we can make this document better to use.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I'm using File Storage (.credentials file) locally so the updated File Storage docs is all I would add. I assumed Vault Storage was ready; if it's not, perhaps best to leave out any details for now. What do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're honestly probably right. We can make reasonable assumptions about how it all ends up being implemented outside MFTF (as a framework), but we can leave out details for now. 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, I've updated the docs to remove the Vault details.

```

<!-- {% endraw %} -->
Expand Down