diff --git a/README.md b/README.md index 6292470..bc69613 100644 --- a/README.md +++ b/README.md @@ -25,8 +25,6 @@ and Nicholas Schrock (@schrockn) from [Facebook](https://www.facebook.com/), the - [Features](#features) - [Examples](#examples) -- [Differences to reference implementation](#differences-to-reference-implementation) - - [Manual dispatching](#manual-dispatching) - [Let's get started!](#lets-get-started) - [Installing](#installing) - [Building](#building) @@ -290,9 +288,136 @@ this was not in place, then all the promises to data will never be dispatched ot See below for more details on `dataLoader.dispatch()` -## Differences to reference implementation +### Error object is not a thing in a type safe Java world + +In the reference JS implementation if the batch loader returns an `Error` object back from the `load()` promise is rejected +with that error. This allows fine grain (per object in the list) sets of error. If I ask for keys A,B,C and B errors out the promise +for B can contain a specific error. + +This is not quite as loose in a Java implementation as Java is a type safe language. + +A batch loader function is defined as `BatchLoader` meaning for a key of type `K` it returns a value of type `V`. + +It cant just return some `Exception` as an object of type `V`. Type safety matters. + +However you can use the `Try` data type which can encapsulate a computation that succeeded or returned an exception. -### Manual dispatching +```java + Try tryS = Try.tryCall(() -> { + if (rollDice()) { + return "OK"; + } else { + throw new RuntimeException("Bang"); + } + }); + + if (tryS.isSuccess()) { + System.out.println("It work " + tryS.get()); + } else { + System.out.println("It failed with exception : " + tryS.getThrowable()); + + } +``` + +DataLoader supports this type and you can use this form to create a batch loader that returns a list of `Try` objects, some of which may have succeeded +and some of which may have failed. From that data loader can infer the right behavior in terms of the `load(x)` promise. + +```java + DataLoader dataLoader = DataLoader.newDataLoaderWithTry(new BatchLoader>() { + @Override + public CompletionStage>> load(List keys) { + return CompletableFuture.supplyAsync(() -> { + List> users = new ArrayList<>(); + for (String key : keys) { + Try userTry = loadUser(key); + users.add(userTry); + } + return users; + }); + } + }); + +``` + +On the above example if one of the `Try` objects represents a failure, then its `load()` promise will complete exceptionally and you can +react to that, in a type safe manner. + + + +## Disabling caching + +In certain uncommon cases, a DataLoader which does not cache may be desirable. + +```java + new DataLoader(userBatchLoader, DataLoaderOptions.newOptions().setCachingEnabled(false)); +``` + +Calling the above will ensure that every call to `.load()` will produce a new promise, and requested keys will not be saved in memory. + +However, when the memoization cache is disabled, your batch function will receive an array of keys which may contain duplicates! Each key will +be associated with each call to `.load()`. Your batch loader should provide a value for each instance of the requested key as per the contract + +```java + userDataLoader.load("A"); + userDataLoader.load("B"); + userDataLoader.load("A"); + + userDataLoader.dispatch(); + + // will result in keys to the batch loader with [ "A", "B", "A" ] + +``` + + +More complex cache behavior can be achieved by calling `.clear()` or `.clearAll()` rather than disabling the cache completely. + + +## Caching errors + +If a batch load fails (that is, a batch function returns a rejected CompletionStage), then the requested values will not be cached. +However if a batch function returns a `Try` or `Throwable` instance for an individual value, then that will be cached to avoid frequently loading +the same problem object. + +In some circumstances you may wish to clear the cache for these individual problems: + +```java + userDataLoader.load("r2d2").whenComplete((user, throwable) -> { + if (throwable != null) { + userDataLoader.clear("r2dr"); + throwable.printStackTrace(); + } else { + processUser(user); + } + }); +``` + +## The scope of a data loader is important + +If you are serving web requests then the data can be specific to the user requesting it. If you have user specific data +then you will not want to cache data meant for user A to then later give it user B in a subsequent request. + +The scope of your `DataLoader` instances is important. You might want to create them per web request to ensure data is only cached within that +web request and no more. + +If your data can be shared across web requests then you might want to scope your data loaders so they survive longer than the web request say. + +## Custom caches + +The default cache behind `DataLoader` is an in memory `HashMap`. There is no expiry on this and it lives for as long as the data loader +lives. + +However you can create your own custom cache and supply it to the data loader on construction via the `org.dataloader.CacheMap` interface. + +```java + MyCustomCache customCache = new MyCustomCache(); + DataLoaderOptions options = DataLoaderOptions.newOptions().setCacheMap(customCache); + new DataLoader(userBatchLoader, options); +``` + +You could choose to use one of the fancy cache implementations from Guava or Kaffeine and wrap it in a `CacheMap` wrapper ready +for data loader. They can do fancy things like time eviction and efficient LRU caching. + +## Manual dispatching The original [Facebook DataLoader](https://github.com/facebook/dataloader) was written in Javascript for NodeJS. NodeJS is single-threaded in nature, but simulates asynchronous logic by invoking functions on separate threads in an event loop, as explained @@ -320,21 +445,6 @@ and there are also gains to this different mode of operation: However, with batch execution control comes responsibility! If you forget to make the call to `dispatch()` then the futures in the load request queue will never be batched, and thus _will never complete_! So be careful when crafting your loader designs. -### Error object is not a thing in a type safe Java world - -In the reference JS implementation if the batch loader returns an `Error` object back then the `loadKey()` promise is rejected -with that error. This allows fine grain (per object in the list) sets of error. If I ask for keys A,B,C and B errors out the promise -for B can contain a specific error. - -This is not quite as neat in a Java implementation - -A batch loader function is defined as `BatchLoader` meaning for a key of type `K` it returns a value of type `V`. - -It cant just return some `Exception` as an object of type `V` since Java is type safe. - -You in order for a batch loader function to return an `Exception` it must be declared as `BatchLoader` which -allows both values and exceptions to be returned . Some type safety is lost in this case if you want -to use the mix of exceptions and values pattern. ## Let's get started! @@ -350,7 +460,7 @@ repositories { } dependencies { - compile 'org.dataloader:java-dataloader:1.0.0' + compile 'com.graphql-java:java-dataloader:1.0.2' } ``` @@ -385,13 +495,13 @@ deal with minor changes. This library was originally written for use within a [VertX world](http://vertx.io/) and it used the vertx-core `Future` classes to implement itself. All the heavy lifting has been done by this project : [vertx-dataloader](https://github.com/engagingspaces/vertx-dataloader) -including the extensive testing. +including the extensive testing (which itself came from Facebook). This particular port was done to reduce the dependency on Vertx and to write a pure Java 8 implementation with no dependencies and also to use the more normative Java CompletableFuture. -[vertx-core](http://vertx.io/docs/vertx-core/java/) is not a lightweight library by any means -so having a pure Java 8 implementation is very desirable. +[vertx-core](http://vertx.io/docs/vertx-core/java/) is not a lightweight library by any means so having a pure Java 8 implementation is +very desirable. This library is entirely inspired by the great works of [Lee Byron](https://github.com/leebyron) and diff --git a/build.gradle b/build.gradle index b2294db..3dc4b40 100644 --- a/build.gradle +++ b/build.gradle @@ -45,7 +45,12 @@ compileJava { sourceCompatibility = 1.8 targetCompatibility = 1.8 - options.compilerArgs = ["-Xlint:unchecked", "-Xdiags:verbose"] + options.compilerArgs = ["-Xlint:unchecked", "-Xdiags:verbose", "-Xdoclint:none"] +} + +task myJavadocs(type: Javadoc) { + source = sourceSets.main.allJava + options.addStringOption('Xdoclint:none', '-quiet') } dependencies { diff --git a/src/main/java/org/dataloader/DataLoader.java b/src/main/java/org/dataloader/DataLoader.java index 911d784..056b8b4 100644 --- a/src/main/java/org/dataloader/DataLoader.java +++ b/src/main/java/org/dataloader/DataLoader.java @@ -38,12 +38,19 @@ * With batching enabled the execution will start after calling {@link DataLoader#dispatch()}, causing the queue of * loaded keys to be sent to the batch function, clears the queue, and returns a promise to the values. *

- * As batch functions are executed the resulting futures are cached using a cache implementation of choice, so they - * will only execute once. Individual cache keys can be cleared, so they will be re-fetched when referred to again. + * As {@link org.dataloader.BatchLoader} batch functions are executed the resulting futures are cached using a cache + * implementation of choice, so they will only execute once. Individual cache keys can be cleared, so they will + * be re-fetched when referred to again. + *

* It is also possible to clear the cache entirely, and prime it with values before they are used. *

* Both caching and batching can be disabled. Configuration of the data loader is done by providing a * {@link DataLoaderOptions} instance on creation. + *

+ * A call to the batch loader might result in individual exception failures for item with the returned list. if + * you want to capture these specific item failures then use {@link org.dataloader.Try} as a return value and + * create the data loader with {@link #newDataLoaderWithTry(BatchLoader)} form. The Try values will be interpreted + * as either success values or cause the {@link #load(Object)} promise to complete exceptionally. * * @param type parameter indicating the type of the data load keys * @param type parameter indicating the type of the data that is returned @@ -58,6 +65,73 @@ public class DataLoader { private final CacheMap> futureCache; private final Map> loaderQueue; + /** + * Creates new DataLoader with the specified batch loader function and default options + * (batching, caching and unlimited batch size). + * + * @param batchLoadFunction the batch load function to use + * @param the key type + * @param the value type + * + * @return a new DataLoader + */ + public static DataLoader newDataLoader(BatchLoader batchLoadFunction) { + return newDataLoader(batchLoadFunction, null); + } + + /** + * Creates new DataLoader with the specified batch loader function with the provided options + * + * @param batchLoadFunction the batch load function to use + * @param options the options to use + * @param the key type + * @param the value type + * + * @return a new DataLoader + */ + public static DataLoader newDataLoader(BatchLoader batchLoadFunction, DataLoaderOptions options) { + return new DataLoader<>(batchLoadFunction, options); + } + + /** + * Creates new DataLoader with the specified batch loader function and default options + * (batching, caching and unlimited batch size) where the batch loader function returns a list of + * {@link org.dataloader.Try} objects. + * + * This allows you to capture both the value that might be returned and also whether exception that might have occurred getting that individual value. If its important you to + * know gther exact status of each item in a batch call and whether it threw exceptions when fetched then + * you can use this form to create the data loader. + * + * @param batchLoadFunction the batch load function to use that uses {@link org.dataloader.Try} objects + * @param the key type + * @param the value type + * + * @return a new DataLoader + */ + public static DataLoader newDataLoaderWithTry(BatchLoader> batchLoadFunction) { + return newDataLoaderWithTry(batchLoadFunction, null); + } + + /** + * Creates new DataLoader with the specified batch loader function and with the provided options + * where the batch loader function returns a list of + * {@link org.dataloader.Try} objects. + * + * @param batchLoadFunction the batch load function to use that uses {@link org.dataloader.Try} objects + * @param options the options to use + * @param the key type + * @param the value type + * + * @return a new DataLoader + * + * @see #newDataLoaderWithTry(BatchLoader) + */ + @SuppressWarnings("unchecked") + public static DataLoader newDataLoaderWithTry(BatchLoader> batchLoadFunction, DataLoaderOptions options) { + return new DataLoader<>((BatchLoader) batchLoadFunction, options); + } + + /** * Creates a new data loader with the provided batch load function, and default options. * @@ -215,6 +289,7 @@ private CompletableFuture> sliceIntoBatchesOfBatches(List keys, List< .collect(Collectors.toList())); } + @SuppressWarnings("unchecked") private CompletableFuture> dispatchQueueBatch(List keys, List> queuedFutures) { return batchLoadFunction.load(keys) .toCompletableFuture() @@ -226,8 +301,13 @@ private CompletableFuture> dispatchQueueBatch(List keys, List future = queuedFutures.get(idx); if (value instanceof Throwable) { future.completeExceptionally((Throwable) value); + // we don't clear the cached view of this entry to avoid + // frequently loading the same error + } else if (value instanceof Try) { + // we allow the batch loader to return a Try so we can better represent a computation + // that might have worked or not. + handleTry((Try) value, future); } else { - @SuppressWarnings("unchecked") V val = (V) value; future.complete(val); } @@ -238,13 +318,21 @@ private CompletableFuture> dispatchQueueBatch(List keys, List future = queuedFutures.get(idx); future.completeExceptionally(ex); - // clear any cached view of this key + // clear any cached view of this key because they all failed clear(key); } return emptyList(); }); } + private void handleTry(Try vTry, CompletableFuture future) { + if (vTry.isSuccess()) { + future.complete(vTry.get()); + } else { + future.completeExceptionally(vTry.getThrowable()); + } + } + /** * Normally {@link #dispatch()} is an asynchronous operation but this version will 'join' on the * results if dispatch and wait for them to complete. If the {@link CompletableFuture} callbacks make more diff --git a/src/main/java/org/dataloader/DataLoaderOptions.java b/src/main/java/org/dataloader/DataLoaderOptions.java index 697bd33..02d10ff 100644 --- a/src/main/java/org/dataloader/DataLoaderOptions.java +++ b/src/main/java/org/dataloader/DataLoaderOptions.java @@ -56,7 +56,10 @@ public DataLoaderOptions(DataLoaderOptions other) { this.maxBatchSize = other.maxBatchSize; } - public static DataLoaderOptions create() { + /** + * @return a new default data loader options that you can then customize + */ + public static DataLoaderOptions newOptions() { return new DataLoaderOptions(); } diff --git a/src/main/java/org/dataloader/DataLoaderRegistry.java b/src/main/java/org/dataloader/DataLoaderRegistry.java index 0617a46..2c11d4c 100644 --- a/src/main/java/org/dataloader/DataLoaderRegistry.java +++ b/src/main/java/org/dataloader/DataLoaderRegistry.java @@ -67,11 +67,13 @@ public DataLoaderRegistry unregister(String key) { * Returns the dataloader that was registered under the specified key * * @param key the key of the data loader + * @param the type of keys + * @param the type of values * * @return a data loader or null if its not present */ + @SuppressWarnings("unchecked") public DataLoader getDataLoader(String key) { - //noinspection unchecked return (DataLoader) dataLoaders.get(key); } diff --git a/src/main/java/org/dataloader/Try.java b/src/main/java/org/dataloader/Try.java new file mode 100644 index 0000000..38ad739 --- /dev/null +++ b/src/main/java/org/dataloader/Try.java @@ -0,0 +1,246 @@ +package org.dataloader; + +import java.util.Optional; +import java.util.concurrent.Callable; +import java.util.concurrent.CompletionStage; +import java.util.function.Consumer; +import java.util.function.Function; +import java.util.function.Supplier; + +import static org.dataloader.impl.Assertions.nonNull; + +/** + * Try is class that allows you to hold the result of computation or the throwable it produced. + * + * This class is useful in {@link org.dataloader.BatchLoader}s so you can mix a batch of calls where some of + * the calls succeeded and some of them failed. You would make your batch loader declaration like : + * + * 

+ * {@code BatchLoader batchLoader = new BatchLoader() { ... } }
+ *
+ * + * {@link org.dataloader.DataLoader} understands the use of Try and will take the exceptional path and complete + * the value promise with that exception value. + */ +public class Try { + private static Throwable NIL = new Throwable() { + @Override + public synchronized Throwable fillInStackTrace() { + return this; + } + }; + + private final Throwable throwable; + private final V value; + + + @SuppressWarnings("unchecked") + private Try(Throwable throwable) { + this.throwable = nonNull(throwable); + this.value = (V) NIL; + } + + private Try(V value) { + this.value = value; + this.throwable = null; + } + + /** + * Creates a Try that has succeeded with the provided value + * + * @param value the successful value + * @param the value type + * + * @return a successful Try + */ + public static Try succeeded(V value) { + return new Try<>(value); + } + + /** + * Creates a Try that has failed with the provided throwable + * + * @param throwable the failed throwable + * @param the value type + * + * @return a failed Try + */ + public static Try failed(Throwable throwable) { + return new Try<>(throwable); + } + + /** + * Calls the callable and if it returns a value, the Try is successful with that value or if throws + * and exception the Try captures that + * + * @param callable the code to call + * @param the value type + * + * @return a Try which is the result of the call + */ + public static Try tryCall(Callable callable) { + try { + return Try.succeeded(callable.call()); + } catch (Exception e) { + return Try.failed(e); + } + } + + /** + * Creates a CompletionStage that, when it completes, will capture into a Try whether the given completionStage + * was successful or not + * + * @param completionStage the completion stage that will complete + * @param the value type + * + * @return a Try which is the result of the call + */ + public static CompletionStage> tryStage(CompletionStage completionStage) { + return completionStage.handle((value, throwable) -> { + if (throwable != null) { + return failed(throwable); + } + return succeeded(value); + }); + } + + /** + * @return the successful value of this try + * + * @throws UnsupportedOperationException if the Try is in fact in the unsuccessful state + */ + public V get() { + if (isFailure()) { + throw new UnsupportedOperationException("You have called Try.get() with a failed Try", throwable); + } + return value; + } + + /** + * @return the failed throwable of this try + * + * @throws UnsupportedOperationException if the Try is in fact in the successful state + */ + public Throwable getThrowable() { + if (isSuccess()) { + throw new UnsupportedOperationException("You have called Try.getThrowable() with a failed Try", throwable); + } + return throwable; + } + + /** + * @return true if this Try succeeded and therefore has a value + */ + public boolean isSuccess() { + return value != NIL; + } + + /** + * @return true if this Try failed and therefore has a throwable + */ + public boolean isFailure() { + return value == NIL; + } + + + /** + * Maps the Try into another Try with a different type + * + * @param mapper the function to map the current Try to a new Try + * @param the target type + * + * @return the mapped Try + */ + public Try map(Function mapper) { + if (isSuccess()) { + return succeeded(mapper.apply(value)); + } + return failed(this.throwable); + } + + /** + * Flats maps the Try into another Try type + * + * @param mapper the flat map function + * @param the target type + * + * @return a new Try + */ + public Try flatMap(Function> mapper) { + if (isSuccess()) { + return mapper.apply(value); + } + return failed(this.throwable); + } + + /** + * Converts the Try into an Optional where unsuccessful tries are empty + * + * @return a new optional + */ + public Optional toOptional() { + return isSuccess() ? Optional.ofNullable(value) : Optional.empty(); + } + + /** + * Returns the successful value of the Try or other if it failed + * + * @param other the other value if the Try failed + * + * @return the value of the Try or an alternative + */ + public V orElse(V other) { + return isSuccess() ? value : other; + } + + /** + * Returns the successful value of the Try or the supplied other if it failed + * + * @param otherSupplier the other value supplied if the Try failed + * + * @return the value of the Try or an alternative + */ + public V orElseGet(Supplier otherSupplier) { + return isSuccess() ? value : otherSupplier.get(); + } + + /** + * Rethrows the underlying throwable inside the unsuccessful Try + * + * @throws Throwable if the Try was in fact throwable + * @throws UnsupportedOperationException if the try was in fact a successful Try + */ + public void reThrow() throws Throwable { + if (isSuccess()) { + throw new UnsupportedOperationException("You have called Try.reThrow() with a successful Try. There is nothing to rethrow"); + } + throw throwable; + } + + /** + * Called the action if the Try has a successful value + * + * @param action the action to call if the Try is successful + */ + void forEach(Consumer action) { + if (isSuccess()) { + action.accept(value); + } + } + + /** + * Called the recover function of the Try failed otherwise returns this if it was successful. + * + * @param recoverFunction the function to recover from a throwable into a new value + * + * @return a Try of the same type + */ + public Try recover(Function recoverFunction) { + if (isFailure()) { + return succeeded(recoverFunction.apply(throwable)); + } + return this; + } + + +} diff --git a/src/test/java/ReadmeExamples.java b/src/test/java/ReadmeExamples.java index 00a8773..ead9d50 100644 --- a/src/test/java/ReadmeExamples.java +++ b/src/test/java/ReadmeExamples.java @@ -3,12 +3,16 @@ import graphql.schema.DataFetchingEnvironment; import graphql.schema.GraphQLSchema; import org.dataloader.BatchLoader; +import org.dataloader.CacheMap; import org.dataloader.DataLoader; +import org.dataloader.DataLoaderOptions; import org.dataloader.DataLoaderRegistry; +import org.dataloader.Try; import org.dataloader.fixtures.User; import org.dataloader.fixtures.UserManager; import org.dataloader.graphql.DataLoaderDispatcherInstrumentation; +import java.util.ArrayList; import java.util.List; import java.util.concurrent.CompletableFuture; import java.util.concurrent.CompletionStage; @@ -154,4 +158,113 @@ private List getCharacterDataViaBatchHTTPApi(List keys) { } + private void tryExample() { + Try tryS = Try.tryCall(() -> { + if (rollDice()) { + return "OK"; + } else { + throw new RuntimeException("Bang"); + } + }); + + if (tryS.isSuccess()) { + System.out.println("It work " + tryS.get()); + } else { + System.out.println("It failed with exception : " + tryS.getThrowable()); + + } + } + + private void tryBatcLoader() { + DataLoader dataLoader = DataLoader.newDataLoaderWithTry(new BatchLoader>() { + @Override + public CompletionStage>> load(List keys) { + return CompletableFuture.supplyAsync(() -> { + List> users = new ArrayList<>(); + for (String key : keys) { + Try userTry = loadUser(key); + users.add(userTry); + } + return users; + }); + } + }); + } + + DataLoader userDataLoader; + + private void clearCacheOnError() { + + userDataLoader.load("r2d2").whenComplete((user, throwable) -> { + if (throwable != null) { + userDataLoader.clear("r2dr"); + throwable.printStackTrace(); + } else { + processUser(user); + } + }); + } + + BatchLoader userBatchLoader; + + private void disableCache() { + new DataLoader(userBatchLoader, DataLoaderOptions.newOptions().setCachingEnabled(false)); + + + userDataLoader.load("A"); + userDataLoader.load("B"); + userDataLoader.load("A"); + + userDataLoader.dispatch(); + + // will result in keys to the batch loader with [ "A", "B", "A" ] + } + + class MyCustomCache implements CacheMap { + @Override + public boolean containsKey(Object key) { + return false; + } + + @Override + public Object get(Object key) { + return null; + } + + @Override + public CacheMap set(Object key, Object value) { + return null; + } + + @Override + public CacheMap delete(Object key) { + return null; + } + + @Override + public CacheMap clear() { + return null; + } + } + + private void customCache() { + + MyCustomCache customCache = new MyCustomCache(); + DataLoaderOptions options = DataLoaderOptions.newOptions().setCacheMap(customCache); + new DataLoader(userBatchLoader, options); + } + + private void processUser(User user) { + + } + + private Try loadUser(String key) { + return null; + } + + private boolean rollDice() { + return false; + } + + } diff --git a/src/test/java/org/dataloader/DataLoaderTest.java b/src/test/java/org/dataloader/DataLoaderTest.java index 7b71ac9..4f9f44c 100644 --- a/src/test/java/org/dataloader/DataLoaderTest.java +++ b/src/test/java/org/dataloader/DataLoaderTest.java @@ -19,6 +19,7 @@ import org.dataloader.fixtures.User; import org.dataloader.fixtures.UserManager; import org.dataloader.impl.CompletableFutureKit; +import org.hamcrest.Matchers; import org.junit.Test; import java.util.ArrayList; @@ -37,12 +38,14 @@ import static java.util.Collections.emptyList; import static java.util.Collections.singletonList; import static org.awaitility.Awaitility.await; +import static org.dataloader.DataLoaderOptions.newOptions; import static org.dataloader.impl.CompletableFutureKit.cause; import static org.dataloader.impl.CompletableFutureKit.failedFuture; import static org.hamcrest.Matchers.empty; import static org.hamcrest.Matchers.equalTo; import static org.hamcrest.Matchers.instanceOf; import static org.hamcrest.Matchers.is; +import static org.hamcrest.Matchers.nullValue; import static org.junit.Assert.assertArrayEquals; import static org.junit.Assert.assertThat; @@ -540,7 +543,7 @@ public void should_Accept_objects_as_keys() { public void should_Disable_caching() throws ExecutionException, InterruptedException { List> loadCalls = new ArrayList<>(); DataLoader identityLoader = - idLoader(DataLoaderOptions.create().setCachingEnabled(false), loadCalls); + idLoader(newOptions().setCachingEnabled(false), loadCalls); CompletableFuture future1 = identityLoader.load("A"); CompletableFuture future2 = identityLoader.load("B"); @@ -578,7 +581,7 @@ public void should_Disable_caching() throws ExecutionException, InterruptedExcep @Test public void should_Accept_objects_with_a_complex_key() throws ExecutionException, InterruptedException { List> loadCalls = new ArrayList<>(); - DataLoaderOptions options = DataLoaderOptions.create().setCacheKeyFunction(getJsonObjectCacheMapFn()); + DataLoaderOptions options = newOptions().setCacheKeyFunction(getJsonObjectCacheMapFn()); DataLoader identityLoader = idLoader(options, loadCalls); JsonObject key1 = new JsonObject().put("id", 123); @@ -599,7 +602,7 @@ public void should_Accept_objects_with_a_complex_key() throws ExecutionException @Test public void should_Clear_objects_with_complex_key() throws ExecutionException, InterruptedException { List> loadCalls = new ArrayList<>(); - DataLoaderOptions options = DataLoaderOptions.create().setCacheKeyFunction(getJsonObjectCacheMapFn()); + DataLoaderOptions options = newOptions().setCacheKeyFunction(getJsonObjectCacheMapFn()); DataLoader identityLoader = idLoader(options, loadCalls); JsonObject key1 = new JsonObject().put("id", 123); @@ -623,7 +626,7 @@ public void should_Clear_objects_with_complex_key() throws ExecutionException, I @Test public void should_Accept_objects_with_different_order_of_keys() throws ExecutionException, InterruptedException { List> loadCalls = new ArrayList<>(); - DataLoaderOptions options = DataLoaderOptions.create().setCacheKeyFunction(getJsonObjectCacheMapFn()); + DataLoaderOptions options = newOptions().setCacheKeyFunction(getJsonObjectCacheMapFn()); DataLoader identityLoader = idLoader(options, loadCalls); JsonObject key1 = new JsonObject().put("a", 123).put("b", 321); @@ -645,7 +648,7 @@ public void should_Accept_objects_with_different_order_of_keys() throws Executio @Test public void should_Allow_priming_the_cache_with_an_object_key() throws ExecutionException, InterruptedException { List> loadCalls = new ArrayList<>(); - DataLoaderOptions options = DataLoaderOptions.create().setCacheKeyFunction(getJsonObjectCacheMapFn()); + DataLoaderOptions options = newOptions().setCacheKeyFunction(getJsonObjectCacheMapFn()); DataLoader identityLoader = idLoader(options, loadCalls); JsonObject key1 = new JsonObject().put("id", 123); @@ -667,7 +670,7 @@ public void should_Allow_priming_the_cache_with_an_object_key() throws Execution public void should_Accept_a_custom_cache_map_implementation() throws ExecutionException, InterruptedException { CustomCacheMap customMap = new CustomCacheMap(); List> loadCalls = new ArrayList<>(); - DataLoaderOptions options = DataLoaderOptions.create().setCacheMap(customMap); + DataLoaderOptions options = newOptions().setCacheMap(customMap); DataLoader identityLoader = idLoader(options, loadCalls); // Fetches as expected @@ -717,7 +720,7 @@ public void should_Accept_a_custom_cache_map_implementation() throws ExecutionEx @Test public void batching_disabled_should_dispatch_immediately() throws Exception { List> loadCalls = new ArrayList<>(); - DataLoaderOptions options = DataLoaderOptions.create().setBatchingEnabled(false); + DataLoaderOptions options = newOptions().setBatchingEnabled(false); DataLoader identityLoader = idLoader(options, loadCalls); CompletableFuture fa = identityLoader.load("A"); @@ -745,7 +748,7 @@ public void batching_disabled_should_dispatch_immediately() throws Exception { @Test public void batching_disabled_and_caching_disabled_should_dispatch_immediately_and_forget() throws Exception { List> loadCalls = new ArrayList<>(); - DataLoaderOptions options = DataLoaderOptions.create().setBatchingEnabled(false).setCachingEnabled(false); + DataLoaderOptions options = newOptions().setBatchingEnabled(false).setCachingEnabled(false); DataLoader identityLoader = idLoader(options, loadCalls); CompletableFuture fa = identityLoader.load("A"); @@ -776,7 +779,7 @@ public void batching_disabled_and_caching_disabled_should_dispatch_immediately_a @Test public void batches_multiple_requests_with_max_batch_size() throws Exception { List> loadCalls = new ArrayList<>(); - DataLoader identityLoader = idLoader(DataLoaderOptions.create().setMaxBatchSize(2), loadCalls); + DataLoader identityLoader = idLoader(newOptions().setMaxBatchSize(2), loadCalls); CompletableFuture f1 = identityLoader.load(1); CompletableFuture f2 = identityLoader.load(2); @@ -797,7 +800,7 @@ public void batches_multiple_requests_with_max_batch_size() throws Exception { @Test public void can_split_max_batch_sizes_correctly() throws Exception { List> loadCalls = new ArrayList<>(); - DataLoader identityLoader = idLoader(DataLoaderOptions.create().setMaxBatchSize(5), loadCalls); + DataLoader identityLoader = idLoader(newOptions().setMaxBatchSize(5), loadCalls); for (int i = 0; i < 21; i++) { identityLoader.load(i); @@ -827,7 +830,7 @@ private Collection listFrom(int i, int max) { @Test public void should_Batch_loads_occurring_within_futures() { List> loadCalls = new ArrayList<>(); - DataLoader identityLoader = idLoader(DataLoaderOptions.create(), loadCalls); + DataLoader identityLoader = idLoader(newOptions(), loadCalls); Supplier nullValue = () -> null; @@ -859,7 +862,7 @@ public void should_Batch_loads_occurring_within_futures() { @Test public void can_call_a_loader_from_a_loader() throws Exception { List> deepLoadCalls = new ArrayList<>(); - DataLoader deepLoader = new DataLoader<>(keys -> { + DataLoader deepLoader = DataLoader.newDataLoader(keys -> { deepLoadCalls.add(keys); return CompletableFuture.completedFuture(keys); }); @@ -939,6 +942,61 @@ public void should_allow_composition_of_data_loader_calls() throws Exception { assertThat(allResults.size(), equalTo(4)); } + @Test + public void should_handle_Trys_coming_back_from_batchLoader() throws Exception { + + List> batchKeyCalls = new ArrayList<>(); + BatchLoader> batchLoader = keys -> { + batchKeyCalls.add(keys); + + List> result = new ArrayList<>(); + for (String key : keys) { + if ("bang".equalsIgnoreCase(key)) { + result.add(Try.failed(new RuntimeException(key))); + } else { + result.add(Try.succeeded(key)); + } + } + return CompletableFuture.completedFuture(result); + }; + + DataLoader dataLoader = DataLoader.newDataLoaderWithTry(batchLoader); + + CompletableFuture a = dataLoader.load("A"); + CompletableFuture b = dataLoader.load("B"); + CompletableFuture bang = dataLoader.load("bang"); + + dataLoader.dispatch(); + + assertThat(a.get(), equalTo("A")); + assertThat(b.get(), equalTo("B")); + assertThat(bang.isCompletedExceptionally(), equalTo(true)); + bang.whenComplete((s, throwable) -> { + assertThat(s, nullValue()); + assertThat(throwable, Matchers.instanceOf(RuntimeException.class)); + assertThat(throwable.getMessage(), equalTo("Bang")); + }); + + assertThat(batchKeyCalls, equalTo(singletonList(asList("A", "B", "bang")))); + + a = dataLoader.load("A"); + b = dataLoader.load("B"); + bang = dataLoader.load("bang"); + + dataLoader.dispatch(); + + assertThat(a.get(), equalTo("A")); + assertThat(b.get(), equalTo("B")); + assertThat(bang.isCompletedExceptionally(), equalTo(true)); + bang.whenComplete((s, throwable) -> { + assertThat(s, nullValue()); + assertThat(throwable, Matchers.instanceOf(RuntimeException.class)); + assertThat(throwable.getMessage(), equalTo("Bang")); + }); + + // the failed value should have been cached as per Facebook DL behaviour + assertThat(batchKeyCalls, equalTo(singletonList(asList("A", "B", "bang")))); + } private static CacheKey getJsonObjectCacheMapFn() { return key -> key.stream() @@ -948,7 +1006,7 @@ private static CacheKey getJsonObjectCacheMapFn() { } private static DataLoader idLoader(DataLoaderOptions options, List> loadCalls) { - return new DataLoader<>(keys -> { + return DataLoader.newDataLoader(keys -> { loadCalls.add(new ArrayList<>(keys)); @SuppressWarnings("unchecked") List values = keys.stream() diff --git a/src/test/java/org/dataloader/TryTest.java b/src/test/java/org/dataloader/TryTest.java new file mode 100644 index 0000000..46514ad --- /dev/null +++ b/src/test/java/org/dataloader/TryTest.java @@ -0,0 +1,196 @@ +package org.dataloader; + +import org.junit.Assert; +import org.junit.Test; + +import java.util.Optional; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.CompletionStage; +import java.util.concurrent.atomic.AtomicReference; +import java.util.function.Function; + +import static org.hamcrest.Matchers.equalTo; +import static org.hamcrest.Matchers.nullValue; +import static org.junit.Assert.assertThat; + +@SuppressWarnings("ConstantConditions") +public class TryTest { + + interface RunThatCanThrow { + void run() throws Throwable; + } + + private void expectThrowable(RunThatCanThrow runnable, Class throwableClass) { + try { + runnable.run(); + } catch (Throwable e) { + if (throwableClass.isInstance(e)) { + return; + } + } + Assert.fail("Expected throwable : " + throwableClass.getName()); + } + + private void assertFailure(Try sTry, String expectedString) { + assertThat(sTry.isSuccess(), equalTo(false)); + assertThat(sTry.isFailure(), equalTo(true)); + assertThat(sTry.getThrowable() instanceof RuntimeException, equalTo(true)); + assertThat(sTry.getThrowable().getMessage(), equalTo(expectedString)); + + expectThrowable(sTry::get, UnsupportedOperationException.class); + } + + private void assertSuccess(Try sTry, String expectedStr) { + assertThat(sTry.isSuccess(), equalTo(true)); + assertThat(sTry.isFailure(), equalTo(false)); + assertThat(sTry.get(), equalTo(expectedStr)); + + expectThrowable(sTry::getThrowable, UnsupportedOperationException.class); + } + + @Test + public void tryFailed() throws Exception { + Try sTry = Try.failed(new RuntimeException("Goodbye Cruel World")); + + assertFailure(sTry, "Goodbye Cruel World"); + } + + @Test + public void trySucceeded() throws Exception { + Try sTry = Try.succeeded("Hello World"); + + assertSuccess(sTry, "Hello World"); + } + + @Test + public void tryCallable() throws Exception { + Try sTry = Try.tryCall(() -> "Hello World"); + + assertSuccess(sTry, "Hello World"); + + sTry = Try.tryCall(() -> { + throw new RuntimeException("Goodbye Cruel World"); + }); + + assertFailure(sTry, "Goodbye Cruel World"); + } + + @Test + public void triedStage() throws Exception { + CompletionStage> sTry = Try.tryStage(CompletableFuture.completedFuture("Hello World")); + + sTry.thenAccept(stageTry -> assertSuccess(stageTry, "Hello World")); + sTry.toCompletableFuture().join(); + + CompletableFuture failure = new CompletableFuture<>(); + failure.completeExceptionally(new RuntimeException("Goodbye Cruel World")); + sTry = Try.tryStage(failure); + + sTry.thenAccept(stageTry -> assertFailure(stageTry, "Goodbye Cruel World")); + sTry.toCompletableFuture().join(); + } + + @Test + public void map() throws Exception { + Try iTry = Try.succeeded(666); + + Try sTry = iTry.map(Object::toString); + assertSuccess(sTry, "666"); + + iTry = Try.failed(new RuntimeException("Goodbye Cruel World")); + + sTry = iTry.map(Object::toString); + assertFailure(sTry, "Goodbye Cruel World"); + } + + @Test + public void flatMap() throws Exception { + Function> intToStringFunc = i -> Try.succeeded(i.toString()); + + Try iTry = Try.succeeded(666); + + Try sTry = iTry.flatMap(intToStringFunc); + assertSuccess(sTry, "666"); + + + iTry = Try.failed(new RuntimeException("Goodbye Cruel World")); + sTry = iTry.flatMap(intToStringFunc); + + assertFailure(sTry, "Goodbye Cruel World"); + + } + + @Test + public void toOptional() throws Exception { + Try iTry = Try.succeeded(666); + Optional optional = iTry.toOptional(); + assertThat(optional.isPresent(), equalTo(true)); + assertThat(optional.get(), equalTo(666)); + + iTry = Try.failed(new RuntimeException("Goodbye Cruel World")); + optional = iTry.toOptional(); + assertThat(optional.isPresent(), equalTo(false)); + } + + @Test + public void orElse() throws Exception { + Try sTry = Try.tryCall(() -> "Hello World"); + + String result = sTry.orElse("other"); + assertThat(result, equalTo("Hello World")); + + sTry = Try.failed(new RuntimeException("Goodbye Cruel World")); + result = sTry.orElse("other"); + assertThat(result, equalTo("other")); + } + + @Test + public void orElseGet() throws Exception { + Try sTry = Try.tryCall(() -> "Hello World"); + + String result = sTry.orElseGet(() -> "other"); + assertThat(result, equalTo("Hello World")); + + sTry = Try.failed(new RuntimeException("Goodbye Cruel World")); + result = sTry.orElseGet(() -> "other"); + assertThat(result, equalTo("other")); + } + + @Test + public void reThrow() throws Exception { + Try sTry = Try.failed(new RuntimeException("Goodbye Cruel World")); + expectThrowable(sTry::reThrow, RuntimeException.class); + + + sTry = Try.tryCall(() -> "Hello World"); + expectThrowable(sTry::reThrow, UnsupportedOperationException.class); + } + + @Test + public void forEach() throws Exception { + AtomicReference sRef = new AtomicReference<>(); + Try sTry = Try.tryCall(() -> "Hello World"); + sTry.forEach(sRef::set); + + assertThat(sRef.get(), equalTo("Hello World")); + + sRef.set(null); + sTry = Try.failed(new RuntimeException("Goodbye Cruel World")); + sTry.forEach(sRef::set); + assertThat(sRef.get(), nullValue()); + } + + @Test + public void recover() throws Exception { + + Try sTry = Try.failed(new RuntimeException("Goodbye Cruel World")); + sTry = sTry.recover(t -> "Hello World"); + + assertSuccess(sTry, "Hello World"); + + sTry = Try.succeeded("Hello Again"); + sTry = sTry.recover(t -> "Hello World"); + + assertSuccess(sTry, "Hello Again"); + } +} \ No newline at end of file diff --git a/src/test/java/org/dataloader/graphql/DataLoaderDispatcherInstrumentationTest.java b/src/test/java/org/dataloader/graphql/DataLoaderDispatcherInstrumentationTest.java index cf8ed72..b5ea43b 100644 --- a/src/test/java/org/dataloader/graphql/DataLoaderDispatcherInstrumentationTest.java +++ b/src/test/java/org/dataloader/graphql/DataLoaderDispatcherInstrumentationTest.java @@ -42,7 +42,7 @@ public void basic_invocation() throws Exception { DataLoaderRegistry registry = new DataLoaderRegistry() .register("a", dlA) .register("b", dlB) - .register("b", dlC); + .register("c", dlC); DataLoaderDispatcherInstrumentation dispatcher = new DataLoaderDispatcherInstrumentation(registry); InstrumentationContext> context = dispatcher.beginExecutionStrategy(null);