Skip to content

feat: deprecate master key providers #242

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Apr 9, 2020
Merged

feat: deprecate master key providers #242

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
4782323
feat: deprecate master key providers
mattsb42-aws Apr 7, 2020
13c3cb2
docs: change main API examples from MKP to keyring
mattsb42-aws Apr 8, 2020
8451c4e
docs: add deprecation notices in MKP docstrings
mattsb42-aws Apr 8, 2020
a7a0b23
docs: add future guidance in MKP deprecation comment
mattsb42-aws Apr 8, 2020
bf6317c
docs: remove examples from readme and replace concepts with dev guide…
mattsb42-aws Apr 8, 2020
b45e928
docs: add link to dev guide concepts section
mattsb42-aws Apr 8, 2020
fc883be
Merge branch 'keyring' into deprecate-mkp
mattsb42-aws Apr 9, 2020
63861e3
docs: update CHANGELOG.rst
mattsb42-aws Apr 9, 2020
a06d4dc
docs: refine wording in MKP removal statements
mattsb42-aws Apr 9, 2020
319cdc5
docs: replace copy-paste concepts descriptions with links to develope…
mattsb42-aws Apr 9, 2020
00b0b6d
Merge branch 'keyring' into deprecate-mkp
mattsb42-aws Apr 9, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions CHANGELOG.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Major Features

* Add `keyrings`_.
* Change one-step APIs to return a :class:`CryptoResult` rather than a tuple.

* Modified APIs: ``aws_encryption_sdk.encrypt`` and ``aws_encryption_sdk.decrypt``.

.. note::
Expand All @@ -20,6 +21,18 @@ Major Features
so this change should not break any existing consumers
unless you are specifically relying on the output being an instance of :class:`tuple`.

Deprecations
------------

* Deprecate master key providers in favor of keyrings.

* We do not have a planned version for when we will remove master key providers.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a little pessimistic, how about something like "Master key providers continue to be supported. Prior to their removal, we will communicate this as defined in our versioning policy."

If that changes, we will communicate that as defined in our versioning policy.

* Deprecate support for Python 3.4.

* This does not mean that this library will no longer work or install with 3.4,
but we are no longer testing against or advertising support 3.4.

1.4.1 -- 2019-09-20
===================
Expand Down
197 changes: 28 additions & 169 deletions README.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ Getting Started
Required Prerequisites
======================

* Python 2.7+ or 3.5+
* Python 2.7 or 3.5+
* cryptography >= 1.8.1
* boto3
* attrs
Expand All @@ -55,189 +55,45 @@ Installation

Concepts
========
There are four main concepts that you need to understand to use this library:
There are three main concepts that are helpful to understand when using the AWS Encryption SDK.

For further information, see the `AWS Encryption SDK developer guide concepts`_.

Cryptographic Materials Managers
--------------------------------
Cryptographic materials managers (CMMs) are resources that collect cryptographic materials and prepare them for
use by the Encryption SDK core logic.

An example of a CMM is the default CMM, which is automatically generated anywhere a caller provides a master
key provider. The default CMM collects encrypted data keys from all master keys referenced by the master key
provider.

An example of a more advanced CMM is the caching CMM, which caches cryptographic materials provided by another CMM.
The cryptographic materials manager (CMM) assembles the cryptographic materials
that are used to encrypt and decrypt data.
The cryptographic materials include plaintext and encrypted data keys, and an optional message signing key.
You can use the default CMM that the AWS Encryption SDK provides or write a custom CMM.
You can specify a CMM, but you never interact with it directly.
The encryption and decryption methods handle it for you.

Master Key Providers
--------------------
Master key providers are resources that provide master keys.
An example of a master key provider is `AWS KMS`_.
The default CMM gets the encryption or decryption materials from
the keyring or master key provider that you specify.
This might involve a call to a cryptographic service, such as AWS Key Management Service (AWS KMS).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure you need to bring this up here, maybe in the keyring section?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This and the descriptions for keyrings and data keys are just copied from the developer guide concepts docs. Thinking this over, to avoid needing to try to keep these in sync, I'm just going to replace these with a single sentence intro and a link to the appropriate section in the dev guide.


To encrypt data in this client, a ``MasterKeyProvider`` object must contain at least one ``MasterKey`` object.
You can specify a CMM and master key provider or keyring, but it's not required.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sentence is a little difficult to parse, I can't tell what is not required. Maybe "You can specify either a CMM, a master key provider, or a keyring"

If you specify a master key provider or keyring, the AWS Encryption SDK creates a Default CMM for you.

``MasterKeyProvider`` objects can also contain other ``MasterKeyProvider`` objects.
Keyrings
--------

Master Keys
-----------
Master keys generate, encrypt, and decrypt data keys.
An example of a master key is a `KMS customer master key (CMK)`_.
A keyring generates, encrypts, and decrypts data keys.
Each keyring is typically associated with a wrapping key or a service that provides and protects wrapping keys.
You can use the keyrings that the AWS Encryption SDK provides or write your own compatible custom keyrings.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what "compatible" is bringing to this sentence


Data Keys
---------
Data keys are the encryption keys that are used to encrypt your data. If your algorithm suite
uses a key derivation function, the data key is used to generate the key that directly encrypts the data.

A data key is an encryption key that the AWS Encryption SDK uses to encrypt your data.
Each data key is a byte array that conforms to the requirements for cryptographic keys.
Unless you're using data key caching, the AWS Encryption SDK uses a unique data key to encrypt each message.

*****
Usage
*****
To use this client, you (the caller) must provide an instance of either a master key provider
or a CMM. The examples in this readme use the ``KMSMasterKeyProvider`` class.

KMSMasterKeyProvider
====================
Because the ``KMSMasterKeyProvider`` uses the `boto3 SDK`_ to interact with `AWS KMS`_, it requires AWS Credentials.
To provide these credentials, use the `standard means by which boto3 locates credentials`_ or provide a
pre-existing instance of a ``botocore session`` to the ``KMSMasterKeyProvider``.
This latter option can be useful if you have an alternate way to store your AWS credentials or
you want to reuse an existing instance of a botocore session in order to decrease startup costs.

.. code:: python

import aws_encryption_sdk
import botocore.session

kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider()

existing_botocore_session = botocore.session.Session()
kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(botocore_session=existing_botocore_session)


You can pre-load the ``KMSMasterKeyProvider`` with one or more CMKs.
To encrypt data, you must configure the ``KMSMasterKeyProvider`` with as least one CMK.
If you configure the the ``KMSMasterKeyProvider`` with multiple CMKs, the `final message`_
will include a copy of the data key encrypted by each configured CMK.

.. code:: python

import aws_encryption_sdk

kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[
'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333'
])

You can add CMKs from multiple regions to the ``KMSMasterKeyProvider``.

.. code:: python

import aws_encryption_sdk

kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[
'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
'arn:aws:kms:us-west-2:3333333333333:key/33333333-3333-3333-3333-333333333333',
'arn:aws:kms:ap-northeast-1:4444444444444:key/44444444-4444-4444-4444-444444444444'
])


Encryption and Decryption
=========================
After you create an instance of a ``MasterKeyProvider``, you can use either of the two
high-level ``encrypt``/``decrypt`` functions to encrypt and decrypt your data.

.. code:: python

import aws_encryption_sdk

kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[
'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333'
])
my_plaintext = b'This is some super secret data! Yup, sure is!'

my_ciphertext, encryptor_header = aws_encryption_sdk.encrypt(
source=my_plaintext,
key_provider=kms_key_provider
)

decrypted_plaintext, decryptor_header = aws_encryption_sdk.decrypt(
source=my_ciphertext,
key_provider=kms_key_provider
)

assert my_plaintext == decrypted_plaintext
assert encryptor_header.encryption_context == decryptor_header.encryption_context

You can provide an `encryption context`_: a form of additional authenticating information.

.. code:: python

import aws_encryption_sdk

kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[
'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333'
])
my_plaintext = b'This is some super secret data! Yup, sure is!'

my_ciphertext, encryptor_header = aws_encryption_sdk.encrypt(
source=my_plaintext,
key_provider=kms_key_provider,
encryption_context={
'not really': 'a secret',
'but adds': 'some authentication'
}
)

decrypted_plaintext, decryptor_header = aws_encryption_sdk.decrypt(
source=my_ciphertext,
key_provider=kms_key_provider
)

assert my_plaintext == decrypted_plaintext
assert encryptor_header.encryption_context == decryptor_header.encryption_context


Streaming
=========
If you are handling large files or simply do not want to put the entire plaintext or ciphertext in
memory at once, you can use this library's streaming clients directly. The streaming clients are
file-like objects, and behave exactly as you would expect a Python file object to behave,
offering context manager and iteration support.

.. code:: python

import aws_encryption_sdk
import filecmp

kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[
'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333'
])
plaintext_filename = 'my-secret-data.dat'
ciphertext_filename = 'my-encrypted-data.ct'

with open(plaintext_filename, 'rb') as pt_file, open(ciphertext_filename, 'wb') as ct_file:
with aws_encryption_sdk.stream(
mode='e',
source=pt_file,
key_provider=kms_key_provider
) as encryptor:
for chunk in encryptor:
ct_file.write(chunk)

new_plaintext_filename = 'my-decrypted-data.dat'

with open(ciphertext_filename, 'rb') as ct_file, open(new_plaintext_filename, 'wb') as pt_file:
with aws_encryption_sdk.stream(
mode='d',
source=ct_file,
key_provider=kms_key_provider
) as decryptor:
for chunk in decryptor:
pt_file.write(chunk)

assert filecmp.cmp(plaintext_filename, new_plaintext_filename)
assert encryptor.header.encryption_context == decryptor.header.encryption_context
For examples of how to use these concepts to accomplish different tasks, see our `examples`_.

Performance Considerations
==========================
Expand All @@ -249,6 +105,8 @@ to your use-case in order to obtain peak performance.


.. _AWS Encryption SDK: https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/introduction.html
.. _AWS Encryption SDK developer guide concepts:
https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/concepts.html
.. _cryptography: https://cryptography.io/en/latest/
.. _cryptography installation guide: https://cryptography.io/en/latest/installation/
.. _Read the Docs: http://aws-encryption-sdk-python.readthedocs.io/en/latest/
Expand All @@ -259,3 +117,4 @@ to your use-case in order to obtain peak performance.
.. _standard means by which boto3 locates credentials: https://boto3.readthedocs.io/en/latest/guide/configuration.html
.. _final message: https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/message-format.html
.. _encryption context: https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#encrypt_context
.. _examples: https://github.com/aws/aws-encryption-sdk-python/tree/master/examples
39 changes: 21 additions & 18 deletions src/aws_encryption_sdk/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,14 +39,15 @@ def encrypt(**kwargs):
.. code:: python

>>> import aws_encryption_sdk
>>> kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[
... 'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
... 'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333'
... ])
>>> from aws_encryption_sdk.keyrings.aws_kms import KmsKeyring
>>> keyring = KmsKeyring(
... generator_key_id="arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222",
... key_ids=["arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333"],
... )
>>> my_ciphertext, encryptor_header = aws_encryption_sdk.encrypt(
... source=my_plaintext,
... key_provider=kms_key_provider
... )
... keyring=keyring,
>>> )

:param config: Client configuration object (config or individual parameters required)
:type config: aws_encryption_sdk.streaming_client.EncryptorConfig
Expand Down Expand Up @@ -106,13 +107,14 @@ def decrypt(**kwargs):
.. code:: python

>>> import aws_encryption_sdk
>>> kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[
... 'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
... 'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333'
... ])
>>> my_ciphertext, encryptor_header = aws_encryption_sdk.decrypt(
>>> from aws_encryption_sdk.keyrings.aws_kms import KmsKeyring
>>> keyring = KmsKeyring(
... generator_key_id="arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222",
... key_ids=["arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333"],
... )
>>> my_ciphertext, decryptor_header = aws_encryption_sdk.decrypt(
... source=my_ciphertext,
... key_provider=kms_key_provider
... keyring=keyring,
... )

:param config: Client configuration object (config or individual parameters required)
Expand Down Expand Up @@ -164,17 +166,18 @@ def stream(**kwargs):
.. code:: python

>>> import aws_encryption_sdk
>>> kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[
... 'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
... 'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333'
... ])
>>> from aws_encryption_sdk.keyrings.aws_kms import KmsKeyring
>>> keyring = KmsKeyring(
... generator_key_id="arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222",
... key_ids=["arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333"],
... )
>>> plaintext_filename = 'my-secret-data.dat'
>>> ciphertext_filename = 'my-encrypted-data.ct'
>>> with open(plaintext_filename, 'rb') as pt_file, open(ciphertext_filename, 'wb') as ct_file:
... with aws_encryption_sdk.stream(
... mode='e',
... source=pt_file,
... key_provider=kms_key_provider
... keyring=keyring,
... ) as encryptor:
... for chunk in encryptor:
... ct_file.write(chunk)
Expand All @@ -183,7 +186,7 @@ def stream(**kwargs):
... with aws_encryption_sdk.stream(
... mode='d',
... source=ct_file,
... key_provider=kms_key_provider
... keyring=keyring,
... ) as decryptor:
... for chunk in decryptor:
... pt_file.write(chunk)
Expand Down
19 changes: 19 additions & 0 deletions src/aws_encryption_sdk/key_providers/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
"""Base class interface for Master Key Providers."""
import abc
import logging
import warnings

import attr
import six
Expand Down Expand Up @@ -51,6 +52,10 @@ class MasterKeyProviderConfig(object):
class MasterKeyProvider(object):
"""Parent interface for Master Key Provider classes.

.. versionadded:: 1.5.0
Master key providers are deprecated.
Use :class:`aws_encryption_sdk.keyrings.base.Keyring` instead.

:param config: Configuration object
:type config: aws_encryption_sdk.key_providers.base.MasterKeyProviderConfig
"""
Expand Down Expand Up @@ -78,6 +83,16 @@ def __new__(cls, **kwargs):
"""Set key index and member set for all new instances here
to avoid requiring child classes to call super init.
"""
# DeprecationWarning are ignored by default,
# but because we do not have a plan to remove master key providers,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like the sound of "we do not have a plan". Maybe just remove these lines and say "Once we decide that we are one X or Y version away from removing master key providers, we should upgrade the DeprecationWarning to a UserWarning so the warning is not ignored by default."

# I think this is the correct level of visibility.
#
# Once we decide that we are one X or Y version away from removing master key providers,
# we should upgrade this to a UserWarning.
warnings.warn(
"Master key providers are deprecated as of 1.5.0. You should migrate to keyrings.", DeprecationWarning
)

instance = super(MasterKeyProvider, cls).__new__(cls)
config = kwargs.pop("config", None)
if not isinstance(config, instance._config_class): # pylint: disable=protected-access
Expand Down Expand Up @@ -329,6 +344,10 @@ def __attrs_post_init__(self):
class MasterKey(MasterKeyProvider):
"""Parent interface for Master Key classes.

.. versionadded:: 1.5.0
Master key providers are deprecated.
Use :class:`aws_encryption_sdk.keyrings.base.Keyring` instead.

:param bytes key_id: Key ID for Master Key
:param config: Configuration object
:type config: aws_encryption_sdk.key_providers.base.MasterKeyConfig
Expand Down
Loading