Skip to content

Commit 0615df4

Browse files
ShaneHarveyShaneHarvey
authored
PYTHON-4112 Support named KMS providers (#1487)
Requires pymongocrypt >= 1.9.0 and libmongocrypt >= 1.9.0.
1 parent 68d22b2 commit 0615df4

File tree

75 files changed

+2674
-86
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

75 files changed

+2674
-86
lines changed

‎doc/changelog.rst

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,12 @@ PyMongo 4.7 brings a number of improvements including:
1414
- Replaced usage of :class:`bson.son.SON` on all internal classes and commands to dict,
1515
:attr:`options.pool_options.metadata` is now of type ``dict`` as opposed to :class:`bson.son.SON`.
1616
- Significantly improved the performance of encoding BSON documents to JSON.
17+
- Support for named KMS providers for client side field level encryption.
18+
Previously supported KMS providers were only: aws, azure, gcp, kmip, and local.
19+
The KMS provider is now expanded to support name suffixes (e.g. local:myname).
20+
Named KMS providers enables more than one of each KMS provider type to be configured.
21+
See the docstring for :class:`~pymongo.encryption_options.AutoEncryptionOpts`.
22+
Note that named KMS providers requires pymongocrypt >=1.9 and libmongocrypt >=1.9.
1723
- Fixed a bug where :class:`~bson.int64.Int64` instances could not always be encoded by `orjson`_. The following now
1824
works::
1925

‎doc/examples/encryption.rst

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -125,8 +125,8 @@ examples show how to setup automatic client-side field level encryption
125125
using :class:`~pymongo.encryption.ClientEncryption` to create a new
126126
encryption data key.
127127

128-
.. note:: Automatic client-side field level encryption requires MongoDB 4.2
129-
enterprise or a MongoDB 4.2 Atlas cluster. The community version of the
128+
.. note:: Automatic client-side field level encryption requires MongoDB >=4.2
129+
enterprise or a MongoDB >=4.2 Atlas cluster. The community version of the
130130
server supports automatic decryption as well as
131131
:ref:`explicit-client-side-encryption`.
132132

@@ -255,7 +255,7 @@ will result in an error.
255255
Server-Side Field Level Encryption Enforcement
256256
``````````````````````````````````````````````
257257

258-
The MongoDB 4.2 server supports using schema validation to enforce encryption
258+
MongoDB >=4.2 servers supports using schema validation to enforce encryption
259259
of specific fields in a collection. This schema validation will prevent an
260260
application from inserting unencrypted values for any fields marked with the
261261
``"encrypt"`` JSON schema keyword.
@@ -457,8 +457,8 @@ Explicit encryption is a MongoDB community feature and does not use the
457457
Explicit Encryption with Automatic Decryption
458458
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
459459

460-
Although automatic encryption requires MongoDB 4.2 enterprise or a
461-
MongoDB 4.2 Atlas cluster, automatic *decryption* is supported for all users.
460+
Although automatic encryption requires MongoDB >=4.2 enterprise or a
461+
MongoDB >=4.2 Atlas cluster, automatic *decryption* is supported for all users.
462462
To configure automatic *decryption* without automatic *encryption* set
463463
``bypass_auto_encryption=True`` in
464464
:class:`~pymongo.encryption_options.AutoEncryptionOpts`:

‎pymongo/encryption.py

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -101,7 +101,7 @@ def _wrap_encryption_errors() -> Iterator[None]:
101101
# we should propagate them unchanged.
102102
raise
103103
except Exception as exc:
104-
raise EncryptionError(exc) from None
104+
raise EncryptionError(exc) from exc
105105

106106

107107
class _EncryptionIO(MongoCryptCallback): # type: ignore[misc]
@@ -520,6 +520,9 @@ def __init__(
520520
data keys. This key should be generated and stored as securely
521521
as possible.
522522
523+
KMS providers may be specified with an optional name suffix
524+
separated by a colon, for example "kmip:name" or "aws:name".
525+
Named KMS providers do not support :ref:`CSFLE on-demand credentials`.
523526
:param key_vault_namespace: The namespace for the key vault collection.
524527
The key vault collection contains all data keys used for encryption
525528
and decryption. Data keys are stored as documents in this MongoDB
@@ -674,12 +677,13 @@ def create_data_key(
674677
"""Create and insert a new data key into the key vault collection.
675678
676679
:param kms_provider: The KMS provider to use. Supported values are
677-
"aws", "azure", "gcp", "kmip", and "local".
680+
"aws", "azure", "gcp", "kmip", "local", or a named provider like
681+
"kmip:name".
678682
:param master_key: Identifies a KMS-specific key used to encrypt the
679683
new data key. If the kmsProvider is "local" the `master_key` is
680684
not applicable and may be omitted.
681685
682-
If the `kms_provider` is "aws" it is required and has the
686+
If the `kms_provider` type is "aws" it is required and has the
683687
following fields::
684688
685689
- `region` (string): Required. The AWS region, e.g. "us-east-1".
@@ -689,15 +693,15 @@ def create_data_key(
689693
requests to. May include port number, e.g.
690694
"kms.us-east-1.amazonaws.com:443".
691695
692-
If the `kms_provider` is "azure" it is required and has the
696+
If the `kms_provider` type is "azure" it is required and has the
693697
following fields::
694698
695699
- `keyVaultEndpoint` (string): Required. Host with optional
696700
port, e.g. "example.vault.azure.net".
697701
- `keyName` (string): Required. Key name in the key vault.
698702
- `keyVersion` (string): Optional. Version of the key to use.
699703
700-
If the `kms_provider` is "gcp" it is required and has the
704+
If the `kms_provider` type is "gcp" it is required and has the
701705
following fields::
702706
703707
- `projectId` (string): Required. The Google cloud project ID.
@@ -709,7 +713,7 @@ def create_data_key(
709713
- `endpoint` (string): Optional. Host with optional port.
710714
Defaults to "cloudkms.googleapis.com".
711715
712-
If the `kms_provider` is "kmip" it is optional and has the
716+
If the `kms_provider` type is "kmip" it is optional and has the
713717
following fields::
714718
715719
- `keyId` (string): Optional. `keyId` is the KMIP Unique

‎pymongo/encryption_options.py

Lines changed: 16 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -55,13 +55,13 @@ def __init__(
5555
) -> None:
5656
"""Options to configure automatic client-side field level encryption.
5757
58-
Automatic client-side field level encryption requires MongoDB 4.2
59-
enterprise or a MongoDB 4.2 Atlas cluster. Automatic encryption is not
58+
Automatic client-side field level encryption requires MongoDB >=4.2
59+
enterprise or a MongoDB >=4.2 Atlas cluster. Automatic encryption is not
6060
supported for operations on a database or view and will result in
6161
error.
6262
63-
Although automatic encryption requires MongoDB 4.2 enterprise or a
64-
MongoDB 4.2 Atlas cluster, automatic *decryption* is supported for all
63+
Although automatic encryption requires MongoDB >=4.2 enterprise or a
64+
MongoDB >=4.2 Atlas cluster, automatic *decryption* is supported for all
6565
users. To configure automatic *decryption* without automatic
6666
*encryption* set ``bypass_auto_encryption=True``. Explicit
6767
encryption and explicit decryption is also supported for all users
@@ -94,12 +94,23 @@ def __init__(
9494
data keys. This key should be generated and stored as securely
9595
as possible.
9696
97+
KMS providers may be specified with an optional name suffix
98+
separated by a colon, for example "kmip:name" or "aws:name".
99+
Named KMS providers do not support :ref:`CSFLE on-demand credentials`.
100+
Named KMS providers enables more than one of each KMS provider type to be configured.
101+
For example, to configure multiple local KMS providers::
102+
103+
kms_providers = {
104+
"local": {"key": local_kek1}, # Unnamed KMS provider.
105+
"local:myname": {"key": local_kek2}, # Named KMS provider with name "myname".
106+
}
107+
97108
:param key_vault_namespace: The namespace for the key vault collection.
98109
The key vault collection contains all data keys used for encryption
99110
and decryption. Data keys are stored as documents in this MongoDB
100111
collection. Data keys are protected with encryption by a KMS
101112
provider.
102-
:param key_vault_client: By default the key vault collection
113+
:param key_vault_client: By default, the key vault collection
103114
is assumed to reside in the same MongoDB cluster as the encrypted
104115
MongoClient. Use this option to route data key queries to a
105116
separate MongoDB cluster.

‎test/__init__.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -113,6 +113,10 @@
113113
"accessKeyId": os.environ.get("FLE_AWS_KEY", ""),
114114
"secretAccessKey": os.environ.get("FLE_AWS_SECRET", ""),
115115
}
116+
AWS_CREDS_2 = {
117+
"accessKeyId": os.environ.get("FLE_AWS_KEY2", ""),
118+
"secretAccessKey": os.environ.get("FLE_AWS_SECRET2", ""),
119+
}
116120
AZURE_CREDS = {
117121
"tenantId": os.environ.get("FLE_AZURE_TENANTID", ""),
118122
"clientId": os.environ.get("FLE_AZURE_CLIENTID", ""),

‎test/client-side-encryption/spec/legacy/fle2v2-BypassQueryAnalysis.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
"runOn": [
33
{
44
"minServerVersion": "7.0.0",
5-
"serverless": "forbid",
65
"topology": [
76
"replicaset",
87
"sharded",

‎test/client-side-encryption/spec/legacy/fle2v2-Compact.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
"runOn": [
33
{
44
"minServerVersion": "7.0.0",
5-
"serverless": "forbid",
65
"topology": [
76
"replicaset",
87
"sharded",

‎test/client-side-encryption/spec/legacy/fle2v2-CreateCollection-OldServer.json

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,38 @@
5555
"result": {
5656
"errorContains": "Driver support of Queryable Encryption is incompatible with server. Upgrade server to use Queryable Encryption."
5757
}
58+
},
59+
{
60+
"name": "assertCollectionNotExists",
61+
"object": "testRunner",
62+
"arguments": {
63+
"database": "default",
64+
"collection": "enxcol_.encryptedCollection.esc"
65+
}
66+
},
67+
{
68+
"name": "assertCollectionNotExists",
69+
"object": "testRunner",
70+
"arguments": {
71+
"database": "default",
72+
"collection": "enxcol_.encryptedCollection.ecc"
73+
}
74+
},
75+
{
76+
"name": "assertCollectionNotExists",
77+
"object": "testRunner",
78+
"arguments": {
79+
"database": "default",
80+
"collection": "enxcol_.encryptedCollection.ecoc"
81+
}
82+
},
83+
{
84+
"name": "assertCollectionNotExists",
85+
"object": "testRunner",
86+
"arguments": {
87+
"database": "default",
88+
"collection": "encryptedCollection"
89+
}
5890
}
5991
]
6092
}

‎test/client-side-encryption/spec/legacy/fle2v2-CreateCollection.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
"runOn": [
33
{
44
"minServerVersion": "7.0.0",
5-
"serverless": "forbid",
65
"topology": [
76
"replicaset",
87
"sharded",

‎test/client-side-encryption/spec/legacy/fle2v2-DecryptExistingData.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
"runOn": [
33
{
44
"minServerVersion": "7.0.0",
5-
"serverless": "forbid",
65
"topology": [
76
"replicaset",
87
"sharded",

‎test/client-side-encryption/spec/legacy/fle2v2-Delete.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
"runOn": [
33
{
44
"minServerVersion": "7.0.0",
5-
"serverless": "forbid",
65
"topology": [
76
"replicaset",
87
"sharded",

‎test/client-side-encryption/spec/legacy/fle2v2-EncryptedFields-vs-EncryptedFieldsMap.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
"runOn": [
33
{
44
"minServerVersion": "7.0.0",
5-
"serverless": "forbid",
65
"topology": [
76
"replicaset",
87
"sharded",

‎test/client-side-encryption/spec/legacy/fle2v2-EncryptedFields-vs-jsonSchema.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
"runOn": [
33
{
44
"minServerVersion": "7.0.0",
5-
"serverless": "forbid",
65
"topology": [
76
"replicaset",
87
"sharded",

‎test/client-side-encryption/spec/legacy/fle2v2-EncryptedFieldsMap-defaults.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
"runOn": [
33
{
44
"minServerVersion": "7.0.0",
5-
"serverless": "forbid",
65
"topology": [
76
"replicaset",
87
"sharded",

‎test/client-side-encryption/spec/legacy/fle2v2-FindOneAndUpdate.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
"runOn": [
33
{
44
"minServerVersion": "7.0.0",
5-
"serverless": "forbid",
65
"topology": [
76
"replicaset",
87
"sharded",

‎test/client-side-encryption/spec/legacy/fle2v2-InsertFind-Indexed.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
"runOn": [
33
{
44
"minServerVersion": "7.0.0",
5-
"serverless": "forbid",
65
"topology": [
76
"replicaset",
87
"sharded",

‎test/client-side-encryption/spec/legacy/fle2v2-InsertFind-Unindexed.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
"runOn": [
33
{
44
"minServerVersion": "7.0.0",
5-
"serverless": "forbid",
65
"topology": [
76
"replicaset",
87
"sharded",

‎test/client-side-encryption/spec/legacy/fle2v2-MissingKey.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
"runOn": [
33
{
44
"minServerVersion": "7.0.0",
5-
"serverless": "forbid",
65
"topology": [
76
"replicaset",
87
"sharded",

‎test/client-side-encryption/spec/legacy/fle2v2-NoEncryption.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
"runOn": [
33
{
44
"minServerVersion": "7.0.0",
5-
"serverless": "forbid",
65
"topology": [
76
"replicaset",
87
"sharded",

‎test/client-side-encryption/spec/legacy/fle2v2-Range-Date-Aggregate.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
"runOn": [
33
{
44
"minServerVersion": "7.0.0",
5-
"serverless": "forbid",
65
"topology": [
76
"replicaset",
87
"sharded",

‎test/client-side-encryption/spec/legacy/fle2v2-Range-Date-Correctness.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
"runOn": [
33
{
44
"minServerVersion": "7.0.0",
5-
"serverless": "forbid",
65
"topology": [
76
"replicaset",
87
"sharded",

‎test/client-side-encryption/spec/legacy/fle2v2-Range-Date-Delete.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
"runOn": [
33
{
44
"minServerVersion": "7.0.0",
5-
"serverless": "forbid",
65
"topology": [
76
"replicaset",
87
"sharded",

‎test/client-side-encryption/spec/legacy/fle2v2-Range-Date-FindOneAndUpdate.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
"runOn": [
33
{
44
"minServerVersion": "7.0.0",
5-
"serverless": "forbid",
65
"topology": [
76
"replicaset",
87
"sharded",

‎test/client-side-encryption/spec/legacy/fle2v2-Range-Date-InsertFind.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
"runOn": [
33
{
44
"minServerVersion": "7.0.0",
5-
"serverless": "forbid",
65
"topology": [
76
"replicaset",
87
"sharded",

‎test/client-side-encryption/spec/legacy/fle2v2-Range-Date-Update.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
"runOn": [
33
{
44
"minServerVersion": "7.0.0",
5-
"serverless": "forbid",
65
"topology": [
76
"replicaset",
87
"sharded",

‎test/client-side-encryption/spec/legacy/fle2v2-Range-Decimal-Aggregate.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
"runOn": [
33
{
44
"minServerVersion": "7.0.0",
5-
"serverless": "forbid",
65
"topology": [
76
"replicaset"
87
]

‎test/client-side-encryption/spec/legacy/fle2v2-Range-Decimal-Correctness.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
"runOn": [
33
{
44
"minServerVersion": "7.0.0",
5-
"serverless": "forbid",
65
"topology": [
76
"replicaset"
87
]

‎test/client-side-encryption/spec/legacy/fle2v2-Range-Decimal-Delete.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
"runOn": [
33
{
44
"minServerVersion": "7.0.0",
5-
"serverless": "forbid",
65
"topology": [
76
"replicaset"
87
]

‎test/client-side-encryption/spec/legacy/fle2v2-Range-Decimal-FindOneAndUpdate.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
"runOn": [
33
{
44
"minServerVersion": "7.0.0",
5-
"serverless": "forbid",
65
"topology": [
76
"replicaset"
87
]

‎test/client-side-encryption/spec/legacy/fle2v2-Range-Decimal-InsertFind.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
"runOn": [
33
{
44
"minServerVersion": "7.0.0",
5-
"serverless": "forbid",
65
"topology": [
76
"replicaset"
87
]

‎test/client-side-encryption/spec/legacy/fle2v2-Range-Decimal-Update.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
"runOn": [
33
{
44
"minServerVersion": "7.0.0",
5-
"serverless": "forbid",
65
"topology": [
76
"replicaset"
87
]

0 commit comments

Comments
 (0)