feat: deprecate master key providers (#242)

* feat: deprecate master key providers

* docs: change main API examples from MKP to keyring

* docs: add deprecation notices in MKP docstrings

* docs: add future guidance in MKP deprecation comment

* docs: remove examples from readme and replace concepts with dev guide concepts

* docs: add link to dev guide concepts section

* docs: update CHANGELOG.rst

Co-Authored-By: Wesley Rosenblum <55108558+WesleyRosenblum@users.noreply.github.com>

* docs: refine wording in MKP removal statements

* docs: replace copy-paste concepts descriptions with links to developer guide

Co-authored-by: Wesley Rosenblum <55108558+WesleyRosenblum@users.noreply.github.com>

Author: mattsb42-aws

CHANGELOG.rst

@@ -10,6 +10,7 @@ Major Features
 
 * Add `keyrings`_.
 * Change one-step APIs to return a :class:`CryptoResult` rather than a tuple.
+
     * Modified APIs: ``aws_encryption_sdk.encrypt`` and ``aws_encryption_sdk.decrypt``.
 
 .. note::
@@ -20,6 +21,19 @@ Major Features
     so this change should not break any existing consumers
     unless you are specifically relying on the output being an instance of :class:`tuple`.
 
+Deprecations
+------------
+
+* Deprecate master key providers in favor of keyrings.
+
+    * We still support using master key providers and are not removing them yet.
+      When we decide to remove them,
+      we will communicate that as defined in our versioning policy.
+
+* Deprecate support for Python 3.4.
+
+    * This does not mean that this library will no longer work or install with 3.4,
+      but we are no longer testing against or advertising support for 3.4.
 
 1.4.1 -- 2019-09-20
 ===================

README.rst

@@ -36,7 +36,7 @@ Getting Started
 Required Prerequisites
 ======================
 
-* Python 2.7+ or 3.5+
+* Python 2.7 or 3.5+
 * cryptography >= 1.8.1
 * boto3
 * attrs
@@ -55,189 +55,42 @@ Installation
 
 Concepts
 ========
-There are four main concepts that you need to understand to use this library:
+There are three main concepts that are helpful to understand when using the AWS Encryption SDK.
+
+For further information, see the `AWS Encryption SDK developer guide concepts`_.
 
 Cryptographic Materials Managers
 --------------------------------
-Cryptographic materials managers (CMMs) are resources that collect cryptographic materials and prepare them for
-use by the Encryption SDK core logic.
-
-An example of a CMM is the default CMM, which is automatically generated anywhere a caller provides a master
-key provider. The default CMM collects encrypted data keys from all master keys referenced by the master key
-provider.
-
-An example of a more advanced CMM is the caching CMM, which caches cryptographic materials provided by another CMM.
+The cryptographic materials manager (CMM) assembles the cryptographic materials
+that are used to encrypt and decrypt data.
 
-Master Key Providers
---------------------
-Master key providers are resources that provide master keys.
-An example of a master key provider is `AWS KMS`_.
+`For more details,
+see the AWS Encryption SDK developer guide cryptographic materials manager concept.
+<https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/concepts.html#crypt-materials-manager>`_
 
-To encrypt data in this client, a ``MasterKeyProvider`` object must contain at least one ``MasterKey`` object.
+Keyrings
+--------
 
-``MasterKeyProvider`` objects can also contain other ``MasterKeyProvider`` objects.
+A keyring generates, encrypts, and decrypts data keys.
 
-Master Keys
------------
-Master keys generate, encrypt, and decrypt data keys.
-An example of a master key is a `KMS customer master key (CMK)`_.
+`For more details,
+see the AWS Encryption SDK developer guide keyring concept.
+<https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/concepts.html#keyring>`_
 
 Data Keys
 ---------
-Data keys are the encryption keys that are used to encrypt your data. If your algorithm suite
-uses a key derivation function, the data key is used to generate the key that directly encrypts the data.
+
+A data key is an encryption key that the AWS Encryption SDK uses to encrypt your data.
+
+`For more details,
+see the AWS Encryption SDK developer guide data key concept.
+<https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/concepts.html#DEK>`_
 
 *****
 Usage
 *****
-To use this client, you (the caller) must provide an instance of either a master key provider
-or a CMM. The examples in this readme use the ``KMSMasterKeyProvider`` class.
-
-KMSMasterKeyProvider
-====================
-Because the ``KMSMasterKeyProvider`` uses the `boto3 SDK`_ to interact with `AWS KMS`_, it requires AWS Credentials.
-To provide these credentials, use the `standard means by which boto3 locates credentials`_ or provide a
-pre-existing instance of a ``botocore session`` to the ``KMSMasterKeyProvider``.
-This latter option can be useful if you have an alternate way to store your AWS credentials or
-you want to reuse an existing instance of a botocore session in order to decrease startup costs.
-
-.. code:: python
-
-    import aws_encryption_sdk
-    import botocore.session
-
-    kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider()
-
-    existing_botocore_session = botocore.session.Session()
-    kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(botocore_session=existing_botocore_session)
-
-
-You can pre-load the ``KMSMasterKeyProvider`` with one or more CMKs.
-To encrypt data, you must configure the ``KMSMasterKeyProvider`` with as least one CMK.
-If you configure the the ``KMSMasterKeyProvider`` with multiple CMKs, the `final message`_
-will include a copy of the data key encrypted by each configured CMK.
-
-.. code:: python
-
-    import aws_encryption_sdk
-
-    kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[
-        'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
-        'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333'
-    ])
-
-You can add CMKs from multiple regions to the ``KMSMasterKeyProvider``.
-
-.. code:: python
-
-    import aws_encryption_sdk
-
-    kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[
-        'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
-        'arn:aws:kms:us-west-2:3333333333333:key/33333333-3333-3333-3333-333333333333',
-        'arn:aws:kms:ap-northeast-1:4444444444444:key/44444444-4444-4444-4444-444444444444'
-    ])
-
-
-Encryption and Decryption
-=========================
-After you create an instance of a ``MasterKeyProvider``, you can use either of the two
-high-level ``encrypt``/``decrypt`` functions to encrypt and decrypt your data.
-
-.. code:: python
-
-    import aws_encryption_sdk
-
-    kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[
-        'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
-        'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333'
-    ])
-    my_plaintext = b'This is some super secret data!  Yup, sure is!'
-
-    my_ciphertext, encryptor_header = aws_encryption_sdk.encrypt(
-        source=my_plaintext,
-        key_provider=kms_key_provider
-    )
-
-    decrypted_plaintext, decryptor_header = aws_encryption_sdk.decrypt(
-        source=my_ciphertext,
-        key_provider=kms_key_provider
-    )
-
-    assert my_plaintext == decrypted_plaintext
-    assert encryptor_header.encryption_context == decryptor_header.encryption_context
-
-You can provide an `encryption context`_: a form of additional authenticating information.
-
-.. code:: python
-
-    import aws_encryption_sdk
-
-    kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[
-        'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
-        'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333'
-    ])
-    my_plaintext = b'This is some super secret data!  Yup, sure is!'
-
-    my_ciphertext, encryptor_header = aws_encryption_sdk.encrypt(
-        source=my_plaintext,
-        key_provider=kms_key_provider,
-        encryption_context={
-            'not really': 'a secret',
-            'but adds': 'some authentication'
-        }
-    )
-
-    decrypted_plaintext, decryptor_header = aws_encryption_sdk.decrypt(
-        source=my_ciphertext,
-        key_provider=kms_key_provider
-    )
-
-    assert my_plaintext == decrypted_plaintext
-    assert encryptor_header.encryption_context == decryptor_header.encryption_context
-
-
-Streaming
-=========
-If you are handling large files or simply do not want to put the entire plaintext or ciphertext in
-memory at once, you can use this library's streaming clients directly. The streaming clients are
-file-like objects, and behave exactly as you would expect a Python file object to behave,
-offering context manager and iteration support.
-
-.. code:: python
-
-    import aws_encryption_sdk
-    import filecmp
-
-    kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[
-        'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
-        'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333'
-    ])
-    plaintext_filename = 'my-secret-data.dat'
-    ciphertext_filename = 'my-encrypted-data.ct'
-
-    with open(plaintext_filename, 'rb') as pt_file, open(ciphertext_filename, 'wb') as ct_file:
-        with aws_encryption_sdk.stream(
-            mode='e',
-            source=pt_file,
-            key_provider=kms_key_provider
-        ) as encryptor:
-            for chunk in encryptor:
-                ct_file.write(chunk)
-
-    new_plaintext_filename = 'my-decrypted-data.dat'
 
-    with open(ciphertext_filename, 'rb') as ct_file, open(new_plaintext_filename, 'wb') as pt_file:
-        with aws_encryption_sdk.stream(
-            mode='d',
-            source=ct_file,
-            key_provider=kms_key_provider
-        ) as decryptor:
-            for chunk in decryptor:
-                pt_file.write(chunk)
-
-    assert filecmp.cmp(plaintext_filename, new_plaintext_filename)
-    assert encryptor.header.encryption_context == decryptor.header.encryption_context
+For examples of how to use these concepts to accomplish different tasks, see our `examples`_.
 
 Performance Considerations
 ==========================
@@ -249,6 +102,8 @@ to your use-case in order to obtain peak performance.
 
 
 .. _AWS Encryption SDK: https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/introduction.html
+.. _AWS Encryption SDK developer guide concepts:
+    https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/concepts.html
 .. _cryptography: https://cryptography.io/en/latest/
 .. _cryptography installation guide: https://cryptography.io/en/latest/installation/
 .. _Read the Docs: http://aws-encryption-sdk-python.readthedocs.io/en/latest/
@@ -259,3 +114,4 @@ to your use-case in order to obtain peak performance.
 .. _standard means by which boto3 locates credentials: https://boto3.readthedocs.io/en/latest/guide/configuration.html
 .. _final message: https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/message-format.html
 .. _encryption context: https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#encrypt_context
+.. _examples: https://github.com/aws/aws-encryption-sdk-python/tree/master/examples

src/aws_encryption_sdk/__init__.py

@@ -39,14 +39,15 @@ def encrypt(**kwargs):
     .. code:: python
 
         >>> import aws_encryption_sdk
-        >>> kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[
-        ...     'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
-        ...     'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333'
-        ... ])
+        >>> from aws_encryption_sdk.keyrings.aws_kms import KmsKeyring
+        >>> keyring = KmsKeyring(
+        ...     generator_key_id="arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222",
+        ...     key_ids=["arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333"],
+        ... )
         >>> my_ciphertext, encryptor_header = aws_encryption_sdk.encrypt(
         ...     source=my_plaintext,
-        ...     key_provider=kms_key_provider
-        ... )
+        ...     keyring=keyring,
+        >>> )
 
     :param config: Client configuration object (config or individual parameters required)
     :type config: aws_encryption_sdk.streaming_client.EncryptorConfig
@@ -106,13 +107,14 @@ def decrypt(**kwargs):
     .. code:: python
 
         >>> import aws_encryption_sdk
-        >>> kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[
-        ...     'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
-        ...     'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333'
-        ... ])
-        >>> my_ciphertext, encryptor_header = aws_encryption_sdk.decrypt(
+        >>> from aws_encryption_sdk.keyrings.aws_kms import KmsKeyring
+        >>> keyring = KmsKeyring(
+        ...     generator_key_id="arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222",
+        ...     key_ids=["arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333"],
+        ... )
+        >>> my_ciphertext, decryptor_header = aws_encryption_sdk.decrypt(
         ...     source=my_ciphertext,
-        ...     key_provider=kms_key_provider
+        ...     keyring=keyring,
         ... )
 
     :param config: Client configuration object (config or individual parameters required)
@@ -164,17 +166,18 @@ def stream(**kwargs):
     .. code:: python
 
         >>> import aws_encryption_sdk
-        >>> kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[
-        ...     'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
-        ...     'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333'
-        ...  ])
+        >>> from aws_encryption_sdk.keyrings.aws_kms import KmsKeyring
+        >>> keyring = KmsKeyring(
+        ...     generator_key_id="arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222",
+        ...     key_ids=["arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333"],
+        ... )
         >>> plaintext_filename = 'my-secret-data.dat'
         >>> ciphertext_filename = 'my-encrypted-data.ct'
         >>> with open(plaintext_filename, 'rb') as pt_file, open(ciphertext_filename, 'wb') as ct_file:
         ...      with aws_encryption_sdk.stream(
         ...         mode='e',
         ...         source=pt_file,
-        ...         key_provider=kms_key_provider
+        ...         keyring=keyring,
         ...     ) as encryptor:
         ...         for chunk in encryptor:
         ...              ct_file.write(chunk)
@@ -183,7 +186,7 @@ def stream(**kwargs):
         ...     with aws_encryption_sdk.stream(
         ...         mode='d',
         ...         source=ct_file,
-        ...         key_provider=kms_key_provider
+        ...         keyring=keyring,
         ...     ) as decryptor:
         ...         for chunk in decryptor:
         ...             pt_file.write(chunk)

src/aws_encryption_sdk/key_providers/base.py

@@ -13,6 +13,7 @@
 """Base class interface for Master Key Providers."""
 import abc
 import logging
+import warnings
 
 import attr
 import six
@@ -51,6 +52,10 @@ class MasterKeyProviderConfig(object):
 class MasterKeyProvider(object):
     """Parent interface for Master Key Provider classes.
 
+    .. versionadded:: 1.5.0
+        Master key providers are deprecated.
+        Use :class:`aws_encryption_sdk.keyrings.base.Keyring` instead.
+
     :param config: Configuration object
     :type config: aws_encryption_sdk.key_providers.base.MasterKeyProviderConfig
     """
@@ -78,6 +83,16 @@ def __new__(cls, **kwargs):
         """Set key index and member set for all new instances here
         to avoid requiring child classes to call super init.
         """
+        # DeprecationWarning are ignored by default,
+        # but because we are not yet removing master key providers,
+        # I think this is the correct level of visibility.
+        #
+        # Once we decide that we are one X or Y version away from removing master key providers,
+        # we should upgrade this to a UserWarning.
+        warnings.warn(
+            "Master key providers are deprecated as of 1.5.0. You should migrate to keyrings.", DeprecationWarning
+        )
+
         instance = super(MasterKeyProvider, cls).__new__(cls)
         config = kwargs.pop("config", None)
         if not isinstance(config, instance._config_class):  # pylint: disable=protected-access
@@ -329,6 +344,10 @@ def __attrs_post_init__(self):
 class MasterKey(MasterKeyProvider):
     """Parent interface for Master Key classes.
 
+    .. versionadded:: 1.5.0
+        Master key providers are deprecated.
+        Use :class:`aws_encryption_sdk.keyrings.base.Keyring` instead.
+
     :param bytes key_id: Key ID for Master Key
     :param config: Configuration object
     :type config: aws_encryption_sdk.key_providers.base.MasterKeyConfig