Skip to content

Introduces a Factory for creating DataLoaders #85

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 26, 2021
Merged

Introduces a Factory for creating DataLoaders #85

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3904878
Introduces a Factory for creating DataLoaders
bbakerman Jun 26, 2021
5573995
Warnings and test bug
bbakerman Jun 26, 2021
d114a40
Doco tweaks
bbakerman Jun 26, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 15 additions & 13 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ and Nicholas Schrock (@schrockn) from [Facebook](https://www.facebook.com/), the
- Deals with partial errors when a batch future fails
- Can disable batching and/or caching in configuration
- Can supply your own [`CacheMap<K, V>`](https://github.com/graphql-java/java-dataloader/blob/master/src/main/java/io/engagingspaces/vertx/dataloader/CacheMap.java) implementations
- Has very high test coverage (see [Acknowledgements](#acknowlegdements))
- Has very high test coverage

## Examples

Expand All @@ -70,7 +70,7 @@ a list of keys
}
};

DataLoader<Long, User> userLoader = DataLoader.newDataLoader(userBatchLoader);
DataLoader<Long, User> userLoader = DataLoaderFactory.newDataLoader(userBatchLoader);

```

Expand Down Expand Up @@ -188,7 +188,7 @@ for the context object.
}
};

DataLoader<String, String> loader = DataLoader.newDataLoader(batchLoader, options);
DataLoader<String, String> loader = DataLoaderFactory.newDataLoader(batchLoader, options);
```

The batch loading code will now receive this environment object and it can be used to get context perhaps allowing it
Expand Down Expand Up @@ -219,7 +219,7 @@ You can gain access to them as a map by key or as the original list of context o
}
};

DataLoader<String, String> loader = DataLoader.newDataLoader(batchLoader, options);
DataLoader<String, String> loader = DataLoaderFactory.newDataLoader(batchLoader, options);
loader.load("keyA", "contextForA");
loader.load("keyB", "contextForB");
```
Expand Down Expand Up @@ -255,7 +255,7 @@ For example, let's assume you want to load users from a database, you could prob
}
};

DataLoader<Long, User> userLoader = DataLoader.newMappedDataLoader(mapBatchLoader);
DataLoader<Long, User> userLoader = DataLoaderFactory.newMappedDataLoader(mapBatchLoader);

// ...
```
Expand Down Expand Up @@ -295,7 +295,7 @@ DataLoader supports this type and you can use this form to create a batch loader
and some of which may have failed. From that data loader can infer the right behavior in terms of the `load(x)` promise.

```java
DataLoader<String, User> dataLoader = DataLoader.newDataLoaderWithTry(new BatchLoader<String, Try<User>>() {
DataLoader<String, User> dataLoader = DataLoaderFactory.newDataLoaderWithTry(new BatchLoader<String, Try<User>>() {
@Override
public CompletionStage<List<Try<User>>> load(List<String> keys) {
return CompletableFuture.supplyAsync(() -> {
Expand All @@ -320,7 +320,7 @@ react to that, in a type safe manner.
In certain uncommon cases, a DataLoader which does not cache may be desirable.

```java
DataLoader.newDataLoader(userBatchLoader, DataLoaderOptions.newOptions().setCachingEnabled(false));
DataLoaderFactory.newDataLoader(userBatchLoader, DataLoaderOptions.newOptions().setCachingEnabled(false));
```

Calling the above will ensure that every call to `.load()` will produce a new promise, and requested keys will not be saved in memory.
Expand Down Expand Up @@ -387,7 +387,7 @@ You can configure the statistics collector used when you build the data loader

```java
DataLoaderOptions options = DataLoaderOptions.newOptions().setStatisticsCollector(() -> new ThreadLocalStatisticsCollector());
DataLoader<String,User> userDataLoader = DataLoader.newDataLoader(userBatchLoader,options);
DataLoader<String,User> userDataLoader = DataLoaderFactory.newDataLoader(userBatchLoader,options);

```

Expand All @@ -399,22 +399,24 @@ and `NoOpStatisticsCollector`.
If you are serving web requests then the data can be specific to the user requesting it. If you have user specific data
then you will not want to cache data meant for user A to then later give it user B in a subsequent request.

The scope of your `DataLoader` instances is important. You might want to create them per web request to ensure data is only cached within that
The scope of your `DataLoader` instances is important. You will want to create them per web request to ensure data is only cached within that
web request and no more.

If your data can be shared across web requests then you might want to scope your data loaders so they survive longer than the web request say.
If your data can be shared across web requests then use a custom cache to keep values in a common place.

Data loaders are stateful components that contain promises (with context) that are likely share the same affinity as the request.

## Custom caches

The default cache behind `DataLoader` is an in memory `HashMap`. There is no expiry on this and it lives for as long as the data loader
The default cache behind `DataLoader` is an in memory `HashMap`. There is no expiry on this, and it lives for as long as the data loader
lives.

However you can create your own custom cache and supply it to the data loader on construction via the `org.dataloader.CacheMap` interface.
However, you can create your own custom cache and supply it to the data loader on construction via the `org.dataloader.CacheMap` interface.

```java
MyCustomCache customCache = new MyCustomCache();
DataLoaderOptions options = DataLoaderOptions.newOptions().setCacheMap(customCache);
DataLoader.newDataLoader(userBatchLoader, options);
DataLoaderFactory.newDataLoader(userBatchLoader, options);
```

You could choose to use one of the fancy cache implementations from Guava or Kaffeine and wrap it in a `CacheMap` wrapper ready
Expand Down
79 changes: 64 additions & 15 deletions src/main/java/org/dataloader/DataLoader.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,10 @@ public class DataLoader<K, V> {
* @param <V> the value type
*
* @return a new DataLoader
*
* @deprecated use {@link DataLoaderFactory} instead
*/
@Deprecated
public static <K, V> DataLoader<K, V> newDataLoader(BatchLoader<K, V> batchLoadFunction) {
return newDataLoader(batchLoadFunction, null);
}
Expand All @@ -89,9 +92,12 @@ public static <K, V> DataLoader<K, V> newDataLoader(BatchLoader<K, V> batchLoadF
* @param <V> the value type
*
* @return a new DataLoader
*
* @deprecated use {@link DataLoaderFactory} instead
*/
@Deprecated
public static <K, V> DataLoader<K, V> newDataLoader(BatchLoader<K, V> batchLoadFunction, DataLoaderOptions options) {
return new DataLoader<>(batchLoadFunction, options);
return DataLoaderFactory.mkDataLoader(batchLoadFunction, options);
}

/**
Expand All @@ -110,7 +116,10 @@ public static <K, V> DataLoader<K, V> newDataLoader(BatchLoader<K, V> batchLoadF
* @param <V> the value type
*
* @return a new DataLoader
*
* @deprecated use {@link DataLoaderFactory} instead
*/
@Deprecated
public static <K, V> DataLoader<K, V> newDataLoaderWithTry(BatchLoader<K, Try<V>> batchLoadFunction) {
return newDataLoaderWithTry(batchLoadFunction, null);
}
Expand All @@ -127,11 +136,12 @@ public static <K, V> DataLoader<K, V> newDataLoaderWithTry(BatchLoader<K, Try<V>
*
* @return a new DataLoader
*
* @see #newDataLoaderWithTry(BatchLoader)
* @see DataLoaderFactory#newDataLoaderWithTry(BatchLoader)
* @deprecated use {@link DataLoaderFactory} instead
*/
@SuppressWarnings("unchecked")
@Deprecated
public static <K, V> DataLoader<K, V> newDataLoaderWithTry(BatchLoader<K, Try<V>> batchLoadFunction, DataLoaderOptions options) {
return new DataLoader<>((BatchLoader<K, V>) batchLoadFunction, options);
return DataLoaderFactory.mkDataLoader(batchLoadFunction, options);
}

/**
Expand All @@ -143,7 +153,10 @@ public static <K, V> DataLoader<K, V> newDataLoaderWithTry(BatchLoader<K, Try<V>
* @param <V> the value type
*
* @return a new DataLoader
*
* @deprecated use {@link DataLoaderFactory} instead
*/
@Deprecated
public static <K, V> DataLoader<K, V> newDataLoader(BatchLoaderWithContext<K, V> batchLoadFunction) {
return newDataLoader(batchLoadFunction, null);
}
Expand All @@ -157,9 +170,12 @@ public static <K, V> DataLoader<K, V> newDataLoader(BatchLoaderWithContext<K, V>
* @param <V> the value type
*
* @return a new DataLoader
*
* @deprecated use {@link DataLoaderFactory} instead
*/
@Deprecated
public static <K, V> DataLoader<K, V> newDataLoader(BatchLoaderWithContext<K, V> batchLoadFunction, DataLoaderOptions options) {
return new DataLoader<>(batchLoadFunction, options);
return DataLoaderFactory.mkDataLoader(batchLoadFunction, options);
}

/**
Expand All @@ -178,7 +194,10 @@ public static <K, V> DataLoader<K, V> newDataLoader(BatchLoaderWithContext<K, V>
* @param <V> the value type
*
* @return a new DataLoader
*
* @deprecated use {@link DataLoaderFactory} instead
*/
@Deprecated
public static <K, V> DataLoader<K, V> newDataLoaderWithTry(BatchLoaderWithContext<K, Try<V>> batchLoadFunction) {
return newDataLoaderWithTry(batchLoadFunction, null);
}
Expand All @@ -195,10 +214,12 @@ public static <K, V> DataLoader<K, V> newDataLoaderWithTry(BatchLoaderWithContex
*
* @return a new DataLoader
*
* @see #newDataLoaderWithTry(BatchLoader)
* @see DataLoaderFactory#newDataLoaderWithTry(BatchLoader)
* @deprecated use {@link DataLoaderFactory} instead
*/
@Deprecated
public static <K, V> DataLoader<K, V> newDataLoaderWithTry(BatchLoaderWithContext<K, Try<V>> batchLoadFunction, DataLoaderOptions options) {
return new DataLoader<>(batchLoadFunction, options);
return DataLoaderFactory.mkDataLoader(batchLoadFunction, options);
}

/**
Expand All @@ -210,7 +231,10 @@ public static <K, V> DataLoader<K, V> newDataLoaderWithTry(BatchLoaderWithContex
* @param <V> the value type
*
* @return a new DataLoader
*
* @deprecated use {@link DataLoaderFactory} instead
*/
@Deprecated
public static <K, V> DataLoader<K, V> newMappedDataLoader(MappedBatchLoader<K, V> batchLoadFunction) {
return newMappedDataLoader(batchLoadFunction, null);
}
Expand All @@ -224,9 +248,12 @@ public static <K, V> DataLoader<K, V> newMappedDataLoader(MappedBatchLoader<K, V
* @param <V> the value type
*
* @return a new DataLoader
*
* @deprecated use {@link DataLoaderFactory} instead
*/
@Deprecated
public static <K, V> DataLoader<K, V> newMappedDataLoader(MappedBatchLoader<K, V> batchLoadFunction, DataLoaderOptions options) {
return new DataLoader<>(batchLoadFunction, options);
return DataLoaderFactory.mkDataLoader(batchLoadFunction, options);
}

/**
Expand All @@ -246,7 +273,10 @@ public static <K, V> DataLoader<K, V> newMappedDataLoader(MappedBatchLoader<K, V
* @param <V> the value type
*
* @return a new DataLoader
*
* @deprecated use {@link DataLoaderFactory} instead
*/
@Deprecated
public static <K, V> DataLoader<K, V> newMappedDataLoaderWithTry(MappedBatchLoader<K, Try<V>> batchLoadFunction) {
return newMappedDataLoaderWithTry(batchLoadFunction, null);
}
Expand All @@ -263,10 +293,12 @@ public static <K, V> DataLoader<K, V> newMappedDataLoaderWithTry(MappedBatchLoad
*
* @return a new DataLoader
*
* @see #newDataLoaderWithTry(BatchLoader)
* @see DataLoaderFactory#newDataLoaderWithTry(BatchLoader)
* @deprecated use {@link DataLoaderFactory} instead
*/
@Deprecated
public static <K, V> DataLoader<K, V> newMappedDataLoaderWithTry(MappedBatchLoader<K, Try<V>> batchLoadFunction, DataLoaderOptions options) {
return new DataLoader<>(batchLoadFunction, options);
return DataLoaderFactory.mkDataLoader(batchLoadFunction, options);
}

/**
Expand All @@ -278,7 +310,10 @@ public static <K, V> DataLoader<K, V> newMappedDataLoaderWithTry(MappedBatchLoad
* @param <V> the value type
*
* @return a new DataLoader
*
* @deprecated use {@link DataLoaderFactory} instead
*/
@Deprecated
public static <K, V> DataLoader<K, V> newMappedDataLoader(MappedBatchLoaderWithContext<K, V> batchLoadFunction) {
return newMappedDataLoader(batchLoadFunction, null);
}
Expand All @@ -292,9 +327,12 @@ public static <K, V> DataLoader<K, V> newMappedDataLoader(MappedBatchLoaderWithC
* @param <V> the value type
*
* @return a new DataLoader
*
* @deprecated use {@link DataLoaderFactory} instead
*/
@Deprecated
public static <K, V> DataLoader<K, V> newMappedDataLoader(MappedBatchLoaderWithContext<K, V> batchLoadFunction, DataLoaderOptions options) {
return new DataLoader<>(batchLoadFunction, options);
return DataLoaderFactory.mkDataLoader(batchLoadFunction, options);
}

/**
Expand All @@ -313,7 +351,10 @@ public static <K, V> DataLoader<K, V> newMappedDataLoader(MappedBatchLoaderWithC
* @param <V> the value type
*
* @return a new DataLoader
*
* @deprecated use {@link DataLoaderFactory} instead
*/
@Deprecated
public static <K, V> DataLoader<K, V> newMappedDataLoaderWithTry(MappedBatchLoaderWithContext<K, Try<V>> batchLoadFunction) {
return newMappedDataLoaderWithTry(batchLoadFunction, null);
}
Expand All @@ -330,32 +371,40 @@ public static <K, V> DataLoader<K, V> newMappedDataLoaderWithTry(MappedBatchLoad
*
* @return a new DataLoader
*
* @see #newDataLoaderWithTry(BatchLoader)
* @see DataLoaderFactory#newDataLoaderWithTry(BatchLoader)
* @deprecated use {@link DataLoaderFactory} instead
*/
@Deprecated
public static <K, V> DataLoader<K, V> newMappedDataLoaderWithTry(MappedBatchLoaderWithContext<K, Try<V>> batchLoadFunction, DataLoaderOptions options) {
return new DataLoader<>(batchLoadFunction, options);
return DataLoaderFactory.mkDataLoader(batchLoadFunction, options);
}

/**
* Creates a new data loader with the provided batch load function, and default options.
*
* @param batchLoadFunction the batch load function to use
*
* @deprecated use {@link DataLoaderFactory} instead
*/
@Deprecated
public DataLoader(BatchLoader<K, V> batchLoadFunction) {
this(batchLoadFunction, null);
this((Object) batchLoadFunction, null);
}

/**
* Creates a new data loader with the provided batch load function and options.
*
* @param batchLoadFunction the batch load function to use
* @param options the batch load options
*
* @deprecated use {@link DataLoaderFactory} instead
*/
@Deprecated
public DataLoader(BatchLoader<K, V> batchLoadFunction, DataLoaderOptions options) {
this((Object) batchLoadFunction, options);
}

private DataLoader(Object batchLoadFunction, DataLoaderOptions options) {
DataLoader(Object batchLoadFunction, DataLoaderOptions options) {
DataLoaderOptions loaderOptions = options == null ? new DataLoaderOptions() : options;
this.futureCache = determineCacheMap(loaderOptions);
// order of keys matter in data loader
Expand Down
Loading