diff --git a/CHANGELOG.rst b/CHANGELOG.rst index c02fb8017..6c51b214d 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -10,6 +10,7 @@ Major Features * Add `keyrings`_. * Change one-step APIs to return a :class:`CryptoResult` rather than a tuple. + * Modified APIs: ``aws_encryption_sdk.encrypt`` and ``aws_encryption_sdk.decrypt``. .. note:: @@ -20,6 +21,19 @@ Major Features so this change should not break any existing consumers unless you are specifically relying on the output being an instance of :class:`tuple`. +Deprecations +------------ + +* Deprecate master key providers in favor of keyrings. + + * We still support using master key providers and are not removing them yet. + When we decide to remove them, + we will communicate that as defined in our versioning policy. + +* Deprecate support for Python 3.4. + + * This does not mean that this library will no longer work or install with 3.4, + but we are no longer testing against or advertising support for 3.4. 1.4.1 -- 2019-09-20 =================== diff --git a/README.rst b/README.rst index ae373f8e3..c0dfb8d8f 100644 --- a/README.rst +++ b/README.rst @@ -36,7 +36,7 @@ Getting Started Required Prerequisites ====================== -* Python 2.7+ or 3.5+ +* Python 2.7 or 3.5+ * cryptography >= 1.8.1 * boto3 * attrs @@ -55,189 +55,42 @@ Installation Concepts ======== -There are four main concepts that you need to understand to use this library: +There are three main concepts that are helpful to understand when using the AWS Encryption SDK. + +For further information, see the `AWS Encryption SDK developer guide concepts`_. Cryptographic Materials Managers -------------------------------- -Cryptographic materials managers (CMMs) are resources that collect cryptographic materials and prepare them for -use by the Encryption SDK core logic. - -An example of a CMM is the default CMM, which is automatically generated anywhere a caller provides a master -key provider. The default CMM collects encrypted data keys from all master keys referenced by the master key -provider. - -An example of a more advanced CMM is the caching CMM, which caches cryptographic materials provided by another CMM. +The cryptographic materials manager (CMM) assembles the cryptographic materials +that are used to encrypt and decrypt data. -Master Key Providers --------------------- -Master key providers are resources that provide master keys. -An example of a master key provider is `AWS KMS`_. +`For more details, +see the AWS Encryption SDK developer guide cryptographic materials manager concept. +`_ -To encrypt data in this client, a ``MasterKeyProvider`` object must contain at least one ``MasterKey`` object. +Keyrings +-------- -``MasterKeyProvider`` objects can also contain other ``MasterKeyProvider`` objects. +A keyring generates, encrypts, and decrypts data keys. -Master Keys ------------ -Master keys generate, encrypt, and decrypt data keys. -An example of a master key is a `KMS customer master key (CMK)`_. +`For more details, +see the AWS Encryption SDK developer guide keyring concept. +`_ Data Keys --------- -Data keys are the encryption keys that are used to encrypt your data. If your algorithm suite -uses a key derivation function, the data key is used to generate the key that directly encrypts the data. + +A data key is an encryption key that the AWS Encryption SDK uses to encrypt your data. + +`For more details, +see the AWS Encryption SDK developer guide data key concept. +`_ ***** Usage ***** -To use this client, you (the caller) must provide an instance of either a master key provider -or a CMM. The examples in this readme use the ``KMSMasterKeyProvider`` class. - -KMSMasterKeyProvider -==================== -Because the ``KMSMasterKeyProvider`` uses the `boto3 SDK`_ to interact with `AWS KMS`_, it requires AWS Credentials. -To provide these credentials, use the `standard means by which boto3 locates credentials`_ or provide a -pre-existing instance of a ``botocore session`` to the ``KMSMasterKeyProvider``. -This latter option can be useful if you have an alternate way to store your AWS credentials or -you want to reuse an existing instance of a botocore session in order to decrease startup costs. - -.. code:: python - - import aws_encryption_sdk - import botocore.session - - kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider() - - existing_botocore_session = botocore.session.Session() - kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(botocore_session=existing_botocore_session) - - -You can pre-load the ``KMSMasterKeyProvider`` with one or more CMKs. -To encrypt data, you must configure the ``KMSMasterKeyProvider`` with as least one CMK. -If you configure the the ``KMSMasterKeyProvider`` with multiple CMKs, the `final message`_ -will include a copy of the data key encrypted by each configured CMK. - -.. code:: python - - import aws_encryption_sdk - - kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[ - 'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222', - 'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333' - ]) - -You can add CMKs from multiple regions to the ``KMSMasterKeyProvider``. - -.. code:: python - - import aws_encryption_sdk - - kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[ - 'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222', - 'arn:aws:kms:us-west-2:3333333333333:key/33333333-3333-3333-3333-333333333333', - 'arn:aws:kms:ap-northeast-1:4444444444444:key/44444444-4444-4444-4444-444444444444' - ]) - - -Encryption and Decryption -========================= -After you create an instance of a ``MasterKeyProvider``, you can use either of the two -high-level ``encrypt``/``decrypt`` functions to encrypt and decrypt your data. - -.. code:: python - - import aws_encryption_sdk - - kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[ - 'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222', - 'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333' - ]) - my_plaintext = b'This is some super secret data! Yup, sure is!' - - my_ciphertext, encryptor_header = aws_encryption_sdk.encrypt( - source=my_plaintext, - key_provider=kms_key_provider - ) - - decrypted_plaintext, decryptor_header = aws_encryption_sdk.decrypt( - source=my_ciphertext, - key_provider=kms_key_provider - ) - - assert my_plaintext == decrypted_plaintext - assert encryptor_header.encryption_context == decryptor_header.encryption_context - -You can provide an `encryption context`_: a form of additional authenticating information. - -.. code:: python - - import aws_encryption_sdk - - kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[ - 'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222', - 'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333' - ]) - my_plaintext = b'This is some super secret data! Yup, sure is!' - - my_ciphertext, encryptor_header = aws_encryption_sdk.encrypt( - source=my_plaintext, - key_provider=kms_key_provider, - encryption_context={ - 'not really': 'a secret', - 'but adds': 'some authentication' - } - ) - - decrypted_plaintext, decryptor_header = aws_encryption_sdk.decrypt( - source=my_ciphertext, - key_provider=kms_key_provider - ) - - assert my_plaintext == decrypted_plaintext - assert encryptor_header.encryption_context == decryptor_header.encryption_context - - -Streaming -========= -If you are handling large files or simply do not want to put the entire plaintext or ciphertext in -memory at once, you can use this library's streaming clients directly. The streaming clients are -file-like objects, and behave exactly as you would expect a Python file object to behave, -offering context manager and iteration support. - -.. code:: python - - import aws_encryption_sdk - import filecmp - - kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[ - 'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222', - 'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333' - ]) - plaintext_filename = 'my-secret-data.dat' - ciphertext_filename = 'my-encrypted-data.ct' - - with open(plaintext_filename, 'rb') as pt_file, open(ciphertext_filename, 'wb') as ct_file: - with aws_encryption_sdk.stream( - mode='e', - source=pt_file, - key_provider=kms_key_provider - ) as encryptor: - for chunk in encryptor: - ct_file.write(chunk) - - new_plaintext_filename = 'my-decrypted-data.dat' - with open(ciphertext_filename, 'rb') as ct_file, open(new_plaintext_filename, 'wb') as pt_file: - with aws_encryption_sdk.stream( - mode='d', - source=ct_file, - key_provider=kms_key_provider - ) as decryptor: - for chunk in decryptor: - pt_file.write(chunk) - - assert filecmp.cmp(plaintext_filename, new_plaintext_filename) - assert encryptor.header.encryption_context == decryptor.header.encryption_context +For examples of how to use these concepts to accomplish different tasks, see our `examples`_. Performance Considerations ========================== @@ -249,6 +102,8 @@ to your use-case in order to obtain peak performance. .. _AWS Encryption SDK: https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/introduction.html +.. _AWS Encryption SDK developer guide concepts: + https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/concepts.html .. _cryptography: https://cryptography.io/en/latest/ .. _cryptography installation guide: https://cryptography.io/en/latest/installation/ .. _Read the Docs: http://aws-encryption-sdk-python.readthedocs.io/en/latest/ @@ -259,3 +114,4 @@ to your use-case in order to obtain peak performance. .. _standard means by which boto3 locates credentials: https://boto3.readthedocs.io/en/latest/guide/configuration.html .. _final message: https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/message-format.html .. _encryption context: https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#encrypt_context +.. _examples: https://github.com/aws/aws-encryption-sdk-python/tree/master/examples diff --git a/src/aws_encryption_sdk/__init__.py b/src/aws_encryption_sdk/__init__.py index cc9b7b9f7..b6a47537c 100644 --- a/src/aws_encryption_sdk/__init__.py +++ b/src/aws_encryption_sdk/__init__.py @@ -39,14 +39,15 @@ def encrypt(**kwargs): .. code:: python >>> import aws_encryption_sdk - >>> kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[ - ... 'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222', - ... 'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333' - ... ]) + >>> from aws_encryption_sdk.keyrings.aws_kms import KmsKeyring + >>> keyring = KmsKeyring( + ... generator_key_id="arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222", + ... key_ids=["arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333"], + ... ) >>> my_ciphertext, encryptor_header = aws_encryption_sdk.encrypt( ... source=my_plaintext, - ... key_provider=kms_key_provider - ... ) + ... keyring=keyring, + >>> ) :param config: Client configuration object (config or individual parameters required) :type config: aws_encryption_sdk.streaming_client.EncryptorConfig @@ -106,13 +107,14 @@ def decrypt(**kwargs): .. code:: python >>> import aws_encryption_sdk - >>> kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[ - ... 'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222', - ... 'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333' - ... ]) - >>> my_ciphertext, encryptor_header = aws_encryption_sdk.decrypt( + >>> from aws_encryption_sdk.keyrings.aws_kms import KmsKeyring + >>> keyring = KmsKeyring( + ... generator_key_id="arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222", + ... key_ids=["arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333"], + ... ) + >>> my_ciphertext, decryptor_header = aws_encryption_sdk.decrypt( ... source=my_ciphertext, - ... key_provider=kms_key_provider + ... keyring=keyring, ... ) :param config: Client configuration object (config or individual parameters required) @@ -164,17 +166,18 @@ def stream(**kwargs): .. code:: python >>> import aws_encryption_sdk - >>> kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[ - ... 'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222', - ... 'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333' - ... ]) + >>> from aws_encryption_sdk.keyrings.aws_kms import KmsKeyring + >>> keyring = KmsKeyring( + ... generator_key_id="arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222", + ... key_ids=["arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333"], + ... ) >>> plaintext_filename = 'my-secret-data.dat' >>> ciphertext_filename = 'my-encrypted-data.ct' >>> with open(plaintext_filename, 'rb') as pt_file, open(ciphertext_filename, 'wb') as ct_file: ... with aws_encryption_sdk.stream( ... mode='e', ... source=pt_file, - ... key_provider=kms_key_provider + ... keyring=keyring, ... ) as encryptor: ... for chunk in encryptor: ... ct_file.write(chunk) @@ -183,7 +186,7 @@ def stream(**kwargs): ... with aws_encryption_sdk.stream( ... mode='d', ... source=ct_file, - ... key_provider=kms_key_provider + ... keyring=keyring, ... ) as decryptor: ... for chunk in decryptor: ... pt_file.write(chunk) diff --git a/src/aws_encryption_sdk/key_providers/base.py b/src/aws_encryption_sdk/key_providers/base.py index 9ee9b7d1c..4ba10585b 100644 --- a/src/aws_encryption_sdk/key_providers/base.py +++ b/src/aws_encryption_sdk/key_providers/base.py @@ -13,6 +13,7 @@ """Base class interface for Master Key Providers.""" import abc import logging +import warnings import attr import six @@ -51,6 +52,10 @@ class MasterKeyProviderConfig(object): class MasterKeyProvider(object): """Parent interface for Master Key Provider classes. + .. versionadded:: 1.5.0 + Master key providers are deprecated. + Use :class:`aws_encryption_sdk.keyrings.base.Keyring` instead. + :param config: Configuration object :type config: aws_encryption_sdk.key_providers.base.MasterKeyProviderConfig """ @@ -78,6 +83,16 @@ def __new__(cls, **kwargs): """Set key index and member set for all new instances here to avoid requiring child classes to call super init. """ + # DeprecationWarning are ignored by default, + # but because we are not yet removing master key providers, + # I think this is the correct level of visibility. + # + # Once we decide that we are one X or Y version away from removing master key providers, + # we should upgrade this to a UserWarning. + warnings.warn( + "Master key providers are deprecated as of 1.5.0. You should migrate to keyrings.", DeprecationWarning + ) + instance = super(MasterKeyProvider, cls).__new__(cls) config = kwargs.pop("config", None) if not isinstance(config, instance._config_class): # pylint: disable=protected-access @@ -329,6 +344,10 @@ def __attrs_post_init__(self): class MasterKey(MasterKeyProvider): """Parent interface for Master Key classes. + .. versionadded:: 1.5.0 + Master key providers are deprecated. + Use :class:`aws_encryption_sdk.keyrings.base.Keyring` instead. + :param bytes key_id: Key ID for Master Key :param config: Configuration object :type config: aws_encryption_sdk.key_providers.base.MasterKeyConfig diff --git a/src/aws_encryption_sdk/key_providers/kms.py b/src/aws_encryption_sdk/key_providers/kms.py index c0a2dc46e..08398b568 100644 --- a/src/aws_encryption_sdk/key_providers/kms.py +++ b/src/aws_encryption_sdk/key_providers/kms.py @@ -78,6 +78,10 @@ class KMSMasterKeyProviderConfig(MasterKeyProviderConfig): class KMSMasterKeyProvider(MasterKeyProvider): """Master Key Provider for KMS. + .. versionadded:: 1.5.0 + Master key providers are deprecated. + Use :class:`aws_encryption_sdk.keyrings.aws_kms.KmsKeyring` instead. + >>> import aws_encryption_sdk >>> kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[ ... 'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222', @@ -226,6 +230,10 @@ def client_default(self): class KMSMasterKey(MasterKey): """Master Key class for KMS CMKs. + .. versionadded:: 1.5.0 + Master key providers are deprecated. + Use :class:`aws_encryption_sdk.keyrings.aws_kms.KmsKeyring` instead. + :param config: Configuration object (config or individual parameters required) :type config: aws_encryption_sdk.key_providers.kms.KMSMasterKeyConfig :param bytes key_id: KMS CMK ID diff --git a/src/aws_encryption_sdk/key_providers/raw.py b/src/aws_encryption_sdk/key_providers/raw.py index 57a1d5edf..fff3487e2 100644 --- a/src/aws_encryption_sdk/key_providers/raw.py +++ b/src/aws_encryption_sdk/key_providers/raw.py @@ -49,6 +49,11 @@ class RawMasterKeyConfig(MasterKeyConfig): class RawMasterKey(MasterKey): """Raw Master Key. + .. versionadded:: 1.5.0 + Master key providers are deprecated. + Use :class:`aws_encryption_sdk.keyrings.raw.RawAESKeyring` + or :class:`aws_encryption_sdk.keyrings.raw.RawRSAKeyring` instead. + :param config: Configuration object (config or individual parameters required) :type config: aws_encryption_sdk.key_providers.raw.RawMasterKeyConfig :param bytes key_id: Key ID for Master Key @@ -192,6 +197,11 @@ def _decrypt_data_key(self, encrypted_data_key, algorithm, encryption_context): class RawMasterKeyProvider(MasterKeyProvider): """Raw Master Key Provider. + .. versionadded:: 1.5.0 + Master key providers are deprecated. + Use :class:`aws_encryption_sdk.keyrings.raw.RawAESKeyring` + or :class:`aws_encryption_sdk.keyrings.raw.RawRSAKeyring` instead. + :param config: Configuration object (optional) :type config: aws_encryption_sdk.key_providers.base.MasterKeyProviderConfig """ diff --git a/test/unit/key_providers/base/test_base_master_key_provider.py b/test/unit/key_providers/base/test_base_master_key_provider.py index 778068dd5..87e9a924c 100644 --- a/test/unit/key_providers/base/test_base_master_key_provider.py +++ b/test/unit/key_providers/base/test_base_master_key_provider.py @@ -60,6 +60,12 @@ def test_repr(): ) +def test_deprecated(): + + with pytest.warns(DeprecationWarning): + MockMasterKeyProvider(provider_id="ex_provider_id", mock_new_master_key="ex_new_master_key") + + class TestBaseMasterKeyProvider(object): def test_provider_id_enforcement(self): class TestProvider(MasterKeyProvider):