Skip to content

Commit 365587d

Browse files
authored
feat: Add DynamoDB provider to parameters module (#1091)
1 parent f8731bf commit 365587d

File tree

8 files changed

+578
-11
lines changed

8 files changed

+578
-11
lines changed

docs/utilities/parameters.md

Lines changed: 57 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,9 @@ description: Utility
55

66

77
The parameters utility provides a way to retrieve parameter values from
8-
[AWS Systems Manager Parameter Store](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html) or
9-
[AWS Secrets Manager](https://aws.amazon.com/secrets-manager/). It also provides a base class to create your parameter provider implementation.
8+
[AWS Systems Manager Parameter Store](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html),
9+
[AWS Secrets Manager](https://aws.amazon.com/secrets-manager/), or [Amazon DynamoDB](https://aws.amazon.com/dynamodb/).
10+
It also provides a base class to create your parameter provider implementation.
1011

1112
**Key features**
1213

@@ -40,11 +41,12 @@ To install this utility, add the following dependency to your project.
4041

4142
This utility requires additional permissions to work as expected. See the table below:
4243

43-
Provider | Function/Method | IAM Permission
44-
------------------------------------------------- | ------------------------------------------------- | ---------------------------------------------------------------------------------
45-
SSM Parameter Store | `SSMProvider.get(String)` `SSMProvider.get(String, Class)` | `ssm:GetParameter`
46-
SSM Parameter Store | `SSMProvider.getMultiple(String)` | `ssm:GetParametersByPath`
47-
Secrets Manager | `SecretsProvider.get(String)` `SecretsProvider.get(String, Class)` | `secretsmanager:GetSecretValue`
44+
Provider | Function/Method | IAM Permission
45+
------------------------------------------------- |----------------------------------------------------------------------| ---------------------------------------------------------------------------------
46+
SSM Parameter Store | `SSMProvider.get(String)` `SSMProvider.get(String, Class)` | `ssm:GetParameter`
47+
SSM Parameter Store | `SSMProvider.getMultiple(String)` | `ssm:GetParametersByPath`
48+
Secrets Manager | `SecretsProvider.get(String)` `SecretsProvider.get(String, Class)` | `secretsmanager:GetSecretValue`
49+
DynamoDB | `DynamoDBProvider.get(String)` `DynamoDBProvider.getMultiple(string)` | `dynamodb:GetItem` `dynamoDB:Query`
4850

4951
## SSM Parameter Store
5052

@@ -74,7 +76,7 @@ in order to get data from other regions or use specific credentials.
7476
}
7577
```
7678

77-
=== "SSMProvider with an explicit region"
79+
=== "SSMProvider with a custom client"
7880

7981
```java hl_lines="5 7"
8082
import software.amazon.lambda.powertools.parameters.SSMProvider;
@@ -149,7 +151,7 @@ in order to get data from other regions or use specific credentials.
149151
}
150152
```
151153

152-
=== "SecretsProvider with an explicit region"
154+
=== "SecretsProvider with a custom client"
153155

154156
```java hl_lines="5 7"
155157
import software.amazon.lambda.powertools.parameters.SecretsProvider;
@@ -166,6 +168,52 @@ in order to get data from other regions or use specific credentials.
166168
}
167169
```
168170

171+
## DynamoDB
172+
To get secrets stored in DynamoDB, use `getDynamoDbProvider`, providing the name of the table that
173+
contains the secrets. As with the other providers, an overloaded methods allows you to retrieve
174+
a `DynamoDbProvider` providing a client if you need to configure it yourself.
175+
176+
=== "DynamoDbProvider"
177+
178+
```java hl_lines="6 9"
179+
import software.amazon.lambda.powertools.parameters.DynamoDbProvider;
180+
import software.amazon.lambda.powertools.parameters.ParamManager;
181+
182+
public class AppWithDynamoDbParameters implements RequestHandler<APIGatewayProxyRequestEvent, APIGatewayProxyResponseEvent> {
183+
// Get an instance of the DynamoDbProvider
184+
DynamoDbProvider ddbProvider = ParamManager.getDynamoDbProvider("my-parameters-table");
185+
186+
// Retrieve a single parameter
187+
String value = ddbProvider.get("my-key");
188+
}
189+
```
190+
191+
=== "DynamoDbProvider with a custom client"
192+
193+
```java hl_lines="9 10 11 12 15 18"
194+
import software.amazon.lambda.powertools.parameters.DynamoDbProvider;
195+
import software.amazon.lambda.powertools.parameters.ParamManager;
196+
import software.amazon.awssdk.services.dynamodb.DynamoDbClient;
197+
import software.amazon.awssdk.http.urlconnection.UrlConnectionHttpClient;
198+
import software.amazon.awssdk.regions.Region;
199+
200+
public class AppWithDynamoDbParameters implements RequestHandler<APIGatewayProxyRequestEvent, APIGatewayProxyResponseEvent> {
201+
// Get a DynamoDB Client with an explicit region
202+
DynamoDbClient ddbClient = DynamoDbClient.builder()
203+
.httpClientBuilder(UrlConnectionHttpClient.builder())
204+
.region(Region.EU_CENTRAL_2)
205+
.build();
206+
207+
// Get an instance of the DynamoDbProvider
208+
DynamoDbProvider provider = ParamManager.getDynamoDbProvider(ddbClient, "test-table");
209+
210+
// Retrieve a single parameter
211+
String value = ddbProvider.get("my-key");
212+
}
213+
```
214+
215+
216+
169217
## Advanced configuration
170218

171219
### Caching

powertools-parameters/pom.xml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,10 @@
7373
<groupId>software.amazon.awssdk</groupId>
7474
<artifactId>url-connection-client</artifactId>
7575
</dependency>
76+
<dependency>
77+
<groupId>software.amazon.awssdk</groupId>
78+
<artifactId>dynamodb</artifactId>
79+
</dependency>
7680
<dependency>
7781
<groupId>com.fasterxml.jackson.core</groupId>
7882
<artifactId>jackson-databind</artifactId>
Lines changed: 191 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,191 @@
1+
package software.amazon.lambda.powertools.parameters;
2+
3+
import software.amazon.awssdk.auth.credentials.EnvironmentVariableCredentialsProvider;
4+
import software.amazon.awssdk.core.SdkSystemSetting;
5+
import software.amazon.awssdk.http.urlconnection.UrlConnectionHttpClient;
6+
import software.amazon.awssdk.regions.Region;
7+
import software.amazon.awssdk.services.dynamodb.DynamoDbClient;
8+
import software.amazon.awssdk.services.dynamodb.model.*;
9+
import software.amazon.lambda.powertools.parameters.cache.CacheManager;
10+
import software.amazon.lambda.powertools.parameters.exception.DynamoDbProviderSchemaException;
11+
import software.amazon.lambda.powertools.parameters.transform.TransformationManager;
12+
13+
import java.util.Collections;
14+
import java.util.Map;
15+
import java.util.stream.Collectors;
16+
17+
/**
18+
* Implements a {@link ParamProvider} on top of DynamoDB. The schema of the table
19+
* is described in the Powertools documentation.
20+
*
21+
* @see <a href="https://awslabs.github.io/aws-lambda-powertools-java/utilities/parameters">Parameters provider documentation</a>
22+
*
23+
*/
24+
public class DynamoDbProvider extends BaseProvider {
25+
26+
private final DynamoDbClient client;
27+
private final String tableName;
28+
29+
public DynamoDbProvider(CacheManager cacheManager, String tableName) {
30+
this(cacheManager, DynamoDbClient.builder()
31+
.httpClientBuilder(UrlConnectionHttpClient.builder())
32+
.credentialsProvider(EnvironmentVariableCredentialsProvider.create())
33+
.region(Region.of(System.getenv(SdkSystemSetting.AWS_REGION.environmentVariable())))
34+
.build(),
35+
tableName
36+
);
37+
38+
}
39+
40+
DynamoDbProvider(CacheManager cacheManager, DynamoDbClient client, String tableName) {
41+
super(cacheManager);
42+
this.client = client;
43+
this.tableName = tableName;
44+
}
45+
46+
/**
47+
* Return a single value from the DynamoDB parameter provider.
48+
*
49+
* @param key key of the parameter
50+
* @return The value, if it exists, null if it doesn't. Throws if the row exists but doesn't match the schema.
51+
*/
52+
@Override
53+
protected String getValue(String key) {
54+
GetItemResponse resp = client.getItem(GetItemRequest.builder()
55+
.tableName(tableName)
56+
.key(Collections.singletonMap("id", AttributeValue.fromS(key)))
57+
.attributesToGet("value")
58+
.build());
59+
60+
// If we have an item at the key, we should be able to get a 'val' out of it. If not it's
61+
// exceptional.
62+
// If we don't have an item at the key, we should return null.
63+
if (resp.hasItem() && !resp.item().values().isEmpty()) {
64+
if (!resp.item().containsKey("value")) {
65+
throw new DynamoDbProviderSchemaException("Missing 'value': " + resp.item().toString());
66+
}
67+
return resp.item().get("value").s();
68+
}
69+
70+
return null;
71+
}
72+
73+
/**
74+
* Returns multiple values from the DynamoDB parameter provider.
75+
*
76+
* @param path Parameter store path
77+
* @return All values matching the given path, and an empty map if none do. Throws if any records exist that don't match the schema.
78+
*/
79+
@Override
80+
protected Map<String, String> getMultipleValues(String path) {
81+
82+
QueryResponse resp = client.query(QueryRequest.builder()
83+
.tableName(tableName)
84+
.keyConditionExpression("id = :v_id")
85+
.expressionAttributeValues(Collections.singletonMap(":v_id", AttributeValue.fromS(path)))
86+
.build());
87+
88+
return resp
89+
.items()
90+
.stream()
91+
.peek((i) -> {
92+
if (!i.containsKey("sk")) {
93+
throw new DynamoDbProviderSchemaException("Missing 'sk': " + i.toString());
94+
}
95+
if (!i.containsKey("value")) {
96+
throw new DynamoDbProviderSchemaException("Missing 'value': " + i.toString());
97+
}
98+
})
99+
.collect(
100+
Collectors.toMap(
101+
(i) -> i.get("sk").s(),
102+
(i) -> i.get("value").s()));
103+
104+
105+
}
106+
107+
/**
108+
* Create a builder that can be used to configure and create a {@link DynamoDbProvider}.
109+
*
110+
* @return a new instance of {@link DynamoDbProvider.Builder}
111+
*/
112+
public static DynamoDbProvider.Builder builder() {
113+
return new DynamoDbProvider.Builder();
114+
}
115+
116+
static class Builder {
117+
private DynamoDbClient client;
118+
private String table;
119+
private CacheManager cacheManager;
120+
private TransformationManager transformationManager;
121+
122+
/**
123+
* Create a {@link DynamoDbProvider} instance.
124+
*
125+
* @return a {@link DynamoDbProvider}
126+
*/
127+
public DynamoDbProvider build() {
128+
if (cacheManager == null) {
129+
throw new IllegalStateException("No CacheManager provided; please provide one");
130+
}
131+
if (table == null) {
132+
throw new IllegalStateException("No DynamoDB table name provided; please provide one");
133+
}
134+
DynamoDbProvider provider;
135+
if (client != null) {
136+
provider = new DynamoDbProvider(cacheManager, client, table);
137+
} else {
138+
provider = new DynamoDbProvider(cacheManager, table);
139+
}
140+
if (transformationManager != null) {
141+
provider.setTransformationManager(transformationManager);
142+
}
143+
return provider;
144+
}
145+
146+
/**
147+
* Set custom {@link DynamoDbClient} to pass to the {@link DynamoDbClient}. <br/>
148+
* Use it if you want to customize the region or any other part of the client.
149+
*
150+
* @param client Custom client
151+
* @return the builder to chain calls (eg. <pre>builder.withClient().build()</pre>)
152+
*/
153+
public DynamoDbProvider.Builder withClient(DynamoDbClient client) {
154+
this.client = client;
155+
return this;
156+
}
157+
158+
/**
159+
* <b>Mandatory</b>. Provide a CacheManager to the {@link DynamoDbProvider}
160+
*
161+
* @param cacheManager the manager that will handle the cache of parameters
162+
* @return the builder to chain calls (eg. <pre>builder.withCacheManager().build()</pre>)
163+
*/
164+
public DynamoDbProvider.Builder withCacheManager(CacheManager cacheManager) {
165+
this.cacheManager = cacheManager;
166+
return this;
167+
}
168+
169+
/**
170+
* <b>Mandatory</b>. Provide a DynamoDB table to the {@link DynamoDbProvider}
171+
*
172+
* @param table the table that parameters will be retrieved from.
173+
* @return the builder to chain calls (eg. <pre>builder.withTable().build()</pre>)
174+
*/
175+
public DynamoDbProvider.Builder withTable(String table) {
176+
this.table = table;
177+
return this;
178+
}
179+
180+
/**
181+
* Provide a transformationManager to the {@link DynamoDbProvider}
182+
*
183+
* @param transformationManager the manager that will handle transformation of parameters
184+
* @return the builder to chain calls (eg. <pre>builder.withTransformationManager().build()</pre>)
185+
*/
186+
public DynamoDbProvider.Builder withTransformationManager(TransformationManager transformationManager) {
187+
this.transformationManager = transformationManager;
188+
return this;
189+
}
190+
}
191+
}

powertools-parameters/src/main/java/software/amazon/lambda/powertools/parameters/ParamManager.java

Lines changed: 32 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@
1515

1616
import software.amazon.awssdk.services.secretsmanager.SecretsManagerClient;
1717
import software.amazon.awssdk.services.ssm.SsmClient;
18+
import software.amazon.awssdk.services.dynamodb.DynamoDbClient;
1819
import software.amazon.lambda.powertools.parameters.cache.CacheManager;
1920
import software.amazon.lambda.powertools.parameters.transform.TransformationManager;
2021

@@ -36,8 +37,8 @@ public final class ParamManager {
3637

3738
/**
3839
* Get a concrete implementation of {@link BaseProvider}.<br/>
39-
* You can specify {@link SecretsProvider} or {@link SSMProvider} or create your custom provider
40-
* by extending {@link BaseProvider} if you need to integrate with a different parameter store.
40+
* You can specify {@link SecretsProvider}, {@link SSMProvider}, {@link DynamoDbProvider}, or create your
41+
* custom provider by extending {@link BaseProvider} if you need to integrate with a different parameter store.
4142
* @return a {@link SecretsProvider}
4243
*/
4344
public static <T extends BaseProvider> T getProvider(Class<T> providerClass) {
@@ -65,6 +66,21 @@ public static SSMProvider getSsmProvider() {
6566
return getProvider(SSMProvider.class);
6667
}
6768

69+
/**
70+
* Get a {@link DynamoDbProvider} with default {@link DynamoDbClient} <br/>
71+
* If you need to customize the region, or other part of the client, use {@link ParamManager#getDynamoDbProvider(DynamoDbClient, String)}
72+
*/
73+
public static DynamoDbProvider getDynamoDbProvider(String tableName) {
74+
// Because we need a DDB table name to configure our client, we can't use
75+
// ParamManager#getProvider. This means that we need to make sure we do the same stuff -
76+
// set transformation manager and cache manager.
77+
return DynamoDbProvider.builder()
78+
.withCacheManager(cacheManager)
79+
.withTable(tableName)
80+
.withTransformationManager(transformationManager)
81+
.build();
82+
}
83+
6884
/**
6985
* Get a {@link SecretsProvider} with your custom {@link SecretsManagerClient}.<br/>
7086
* Use this to configure region or other part of the client. Use {@link ParamManager#getSsmProvider()} if you don't need this customization.
@@ -91,6 +107,20 @@ public static SSMProvider getSsmProvider(SsmClient client) {
91107
.build());
92108
}
93109

110+
/**
111+
* Get a {@link DynamoDbProvider} with your custom {@link DynamoDbClient}.<br/>
112+
* Use this to configure region or other part of the client. Use {@link ParamManager#getDynamoDbProvider(String)} )} if you don't need this customization.
113+
* @return a {@link DynamoDbProvider}
114+
*/
115+
public static DynamoDbProvider getDynamoDbProvider(DynamoDbClient client, String table) {
116+
return (DynamoDbProvider) providers.computeIfAbsent(DynamoDbProvider.class, (k) -> DynamoDbProvider.builder()
117+
.withClient(client)
118+
.withTable(table)
119+
.withCacheManager(cacheManager)
120+
.withTransformationManager(transformationManager)
121+
.build());
122+
}
123+
94124
public static CacheManager getCacheManager() {
95125
return cacheManager;
96126
}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
package software.amazon.lambda.powertools.parameters.exception;
2+
3+
/**
4+
* Thrown when the DynamoDbProvider comes across parameter data that
5+
* does not meet the DynamoDB parameters schema.
6+
*/
7+
public class DynamoDbProviderSchemaException extends RuntimeException {
8+
public DynamoDbProviderSchemaException(String msg) {
9+
super(msg);
10+
}
11+
}

0 commit comments

Comments
 (0)