diff --git a/README.md b/README.md index 73351ad..4263dff 100644 --- a/README.md +++ b/README.md @@ -144,7 +144,7 @@ a list of user ids in one call. This is important consideration. By using `dataloader` you have batched up the requests for N keys in a list of keys that can be retrieved at one time. - If you don't have batched backing services, then you cant be as efficient as possible as you will have to make N calls for each key. + If you don't have batched backing services, then you can't be as efficient as possible as you will have to make N calls for each key. ```java BatchLoader lessEfficientUserBatchLoader = new BatchLoader() { @@ -313,6 +313,49 @@ and some of which may have failed. From that data loader can infer the right be On the above example if one of the `Try` objects represents a failure, then its `load()` promise will complete exceptionally and you can react to that, in a type safe manner. +## Caching + +`DataLoader` has a two tiered caching system in place. + +The first cache is represented by the interface `org.dataloader.CacheMap`. It will cache `CompletableFuture`s by key and hence future `load(key)` calls +will be given the same future and hence the same value. + +This cache can only work local to the JVM, since its caches `CompletableFuture`s which cannot be serialised across a network say. + +The second level cache is a value cache represented by the interface `org.dataloader.ValueCache`. By default, this is not enabled and is a no-op. + +The value cache uses an async API pattern to encapsulate the idea that the value cache could be in a remote place such as REDIS or Memcached. + +## Custom future caches + +The default future cache behind `DataLoader` is an in memory `HashMap`. There is no expiry on this, and it lives for as long as the data loader +lives. + +However, you can create your own custom cache and supply it to the data loader on construction via the `org.dataloader.CacheMap` interface. + +```java + MyCustomCache customCache = new MyCustomCache(); + DataLoaderOptions options = DataLoaderOptions.newOptions().setCacheMap(customCache); + DataLoaderFactory.newDataLoader(userBatchLoader, options); +``` + +You could choose to use one of the fancy cache implementations from Guava or Caffeine and wrap it in a `CacheMap` wrapper ready +for data loader. They can do fancy things like time eviction and efficient LRU caching. + +As stated above, a custom `org.dataloader.CacheMap` is a local cache of futures with values, not values per se. + +## Custom value caches + +You will need to create your own implementations of the `org.dataloader.ValueCache` if your want to use an external cache. + +This library does not ship with any implementations of `ValueCache` because it does not want to have +production dependencies on external cache libraries, but you can easily write your own. + +The tests have an example based on [Caffeine](https://github.com/ben-manes/caffeine). + +The API of `ValueCache` has been designed to be asynchronous because it is expected that the value cache could be outside +your JVM. It uses `CompleteableFuture`s to get and set values into cache, which may involve a network call and hence exceptional failures to get +or set values. ## Disabling caching @@ -346,7 +389,7 @@ More complex cache behavior can be achieved by calling `.clear()` or `.clearAll( ## Caching errors If a batch load fails (that is, a batch function returns a rejected CompletionStage), then the requested values will not be cached. -However if a batch function returns a `Try` or `Throwable` instance for an individual value, then that will be cached to avoid frequently loading +However, if a batch function returns a `Try` or `Throwable` instance for an individual value, then that will be cached to avoid frequently loading the same problem object. In some circumstances you may wish to clear the cache for these individual problems: @@ -406,33 +449,18 @@ If your data can be shared across web requests then use a custom cache to keep v Data loaders are stateful components that contain promises (with context) that are likely share the same affinity as the request. -## Custom caches - -The default cache behind `DataLoader` is an in memory `HashMap`. There is no expiry on this, and it lives for as long as the data loader -lives. - -However, you can create your own custom cache and supply it to the data loader on construction via the `org.dataloader.CacheMap` interface. - -```java - MyCustomCache customCache = new MyCustomCache(); - DataLoaderOptions options = DataLoaderOptions.newOptions().setCacheMap(customCache); - DataLoaderFactory.newDataLoader(userBatchLoader, options); -``` - -You could choose to use one of the fancy cache implementations from Guava or Kaffeine and wrap it in a `CacheMap` wrapper ready -for data loader. They can do fancy things like time eviction and efficient LRU caching. - ## Manual dispatching -The original [Facebook DataLoader](https://github.com/facebook/dataloader) was written in Javascript for NodeJS. NodeJS is single-threaded in nature, but simulates -asynchronous logic by invoking functions on separate threads in an event loop, as explained +The original [Facebook DataLoader](https://github.com/facebook/dataloader) was written in Javascript for NodeJS. + +NodeJS is single-threaded in nature, but simulates asynchronous logic by invoking functions on separate threads in an event loop, as explained [in this post](http://stackoverflow.com/a/19823583/3455094) on StackOverflow. NodeJS generates so-call 'ticks' in which queued functions are dispatched for execution, and Facebook `DataLoader` uses the `nextTick()` function in NodeJS to _automatically_ dequeue load requests and send them to the batch execution function for processing. -And here there is an **IMPORTANT DIFFERENCE** compared to how `java-dataloader` operates!! +Here there is an **IMPORTANT DIFFERENCE** compared to how `java-dataloader` operates!! In NodeJS the batch preparation will not affect the asynchronous processing behaviour in any way. It will just prepare batches in 'spare time' as it were. diff --git a/build.gradle b/build.gradle index 86004bd..cdbca84 100644 --- a/build.gradle +++ b/build.gradle @@ -58,6 +58,7 @@ dependencies { testCompile 'org.slf4j:slf4j-simple:' + slf4jVersion testCompile "junit:junit:4.12" testCompile 'org.awaitility:awaitility:2.0.0' + testImplementation 'com.github.ben-manes.caffeine:caffeine:2.9.0' } task sourcesJar(type: Jar) { diff --git a/src/main/java/org/dataloader/CacheMap.java b/src/main/java/org/dataloader/CacheMap.java index 9373a77..0db31b8 100644 --- a/src/main/java/org/dataloader/CacheMap.java +++ b/src/main/java/org/dataloader/CacheMap.java @@ -22,43 +22,44 @@ import java.util.concurrent.CompletableFuture; /** - * Cache map interface for data loaders that use caching. + * CacheMap is used by data loaders that use caching promises to values aka {@link CompletableFuture}<V>. A better name for this + * class might have been FutureCache but that is history now. *

- * The default implementation used by the data loader is based on a {@link java.util.LinkedHashMap}. Note that the - * implementation could also have used a regular {@link java.util.Map} instead of this {@link CacheMap}, but - * this aligns better to the reference data loader implementation provided by Facebook + * The default implementation used by the data loader is based on a {@link java.util.LinkedHashMap}. *

- * Also it doesn't require you to implement the full set of map overloads, just the required methods. + * This is really a cache of completed {@link CompletableFuture}<V> values in memory. It is used, when caching is enabled, to + * give back the same future to any code that may call it. If you need a cache of the underlying values that is possible external to the JVM + * then you will want to use {{@link ValueCache}} which is designed for external cache access. * - * @param type parameter indicating the type of the cache keys + * @param type parameter indicating the type of the cache keys * @param type parameter indicating the type of the data that is cached * * @author Arnold Schrijver * @author Brad Baker */ @PublicSpi -public interface CacheMap { +public interface CacheMap { /** * Creates a new cache map, using the default implementation that is based on a {@link java.util.LinkedHashMap}. * - * @param type parameter indicating the type of the cache keys + * @param type parameter indicating the type of the cache keys * @param type parameter indicating the type of the data that is cached * * @return the cache map */ - static CacheMap> simpleMap() { + static CacheMap simpleMap() { return new DefaultCacheMap<>(); } /** - * Checks whether the specified key is contained in the cach map. + * Checks whether the specified key is contained in the cache map. * * @param key the key to check * * @return {@code true} if the cache contains the key, {@code false} otherwise */ - boolean containsKey(U key); + boolean containsKey(K key); /** * Gets the specified key from the cache map. @@ -70,7 +71,7 @@ static CacheMap> simpleMap() { * * @return the cached value, or {@code null} if not found (depends on cache implementation) */ - V get(U key); + CompletableFuture get(K key); /** * Creates a new cache map entry with the specified key and value, or updates the value if the key already exists. @@ -80,7 +81,7 @@ static CacheMap> simpleMap() { * * @return the cache map for fluent coding */ - CacheMap set(U key, V value); + CacheMap set(K key, CompletableFuture value); /** * Deletes the entry with the specified key from the cache map, if it exists. @@ -89,12 +90,12 @@ static CacheMap> simpleMap() { * * @return the cache map for fluent coding */ - CacheMap delete(U key); + CacheMap delete(K key); /** * Clears all entries of the cache map * * @return the cache map for fluent coding */ - CacheMap clear(); + CacheMap clear(); } diff --git a/src/main/java/org/dataloader/DataLoader.java b/src/main/java/org/dataloader/DataLoader.java index 18e7900..0aea1c7 100644 --- a/src/main/java/org/dataloader/DataLoader.java +++ b/src/main/java/org/dataloader/DataLoader.java @@ -30,6 +30,7 @@ import java.util.List; import java.util.Optional; import java.util.concurrent.CompletableFuture; +import java.util.function.BiConsumer; import static org.dataloader.impl.Assertions.nonNull; @@ -64,8 +65,9 @@ public class DataLoader { private final DataLoaderHelper helper; - private final CacheMap> futureCache; private final StatisticsCollector stats; + private final CacheMap futureCache; + private final ValueCache valueCache; /** * Creates new DataLoader with the specified batch loader function and default options @@ -413,19 +415,24 @@ public DataLoader(BatchLoader batchLoadFunction, DataLoaderOptions options @VisibleForTesting DataLoader(Object batchLoadFunction, DataLoaderOptions options, Clock clock) { DataLoaderOptions loaderOptions = options == null ? new DataLoaderOptions() : options; - this.futureCache = determineCacheMap(loaderOptions); + this.futureCache = determineFutureCache(loaderOptions); + this.valueCache = determineValueCache(loaderOptions); // order of keys matter in data loader this.stats = nonNull(loaderOptions.getStatisticsCollector()); - this.helper = new DataLoaderHelper<>(this, batchLoadFunction, loaderOptions, this.futureCache, this.stats, clock); + this.helper = new DataLoaderHelper<>(this, batchLoadFunction, loaderOptions, this.futureCache, this.valueCache, this.stats, clock); } @SuppressWarnings("unchecked") - private CacheMap> determineCacheMap(DataLoaderOptions loaderOptions) { - return loaderOptions.cacheMap().isPresent() ? (CacheMap>) loaderOptions.cacheMap().get() : CacheMap.simpleMap(); + private CacheMap determineFutureCache(DataLoaderOptions loaderOptions) { + return (CacheMap) loaderOptions.cacheMap().orElseGet(CacheMap::simpleMap); } + @SuppressWarnings("unchecked") + private ValueCache determineValueCache(DataLoaderOptions loaderOptions) { + return (ValueCache) loaderOptions.valueCache().orElseGet(ValueCache::defaultValueCache); + } /** * This returns the last instant the data loader was dispatched. When the data loader is created this value is set to now. @@ -628,9 +635,24 @@ public int dispatchDepth() { * @return the data loader for fluent coding */ public DataLoader clear(K key) { + return clear(key, (v, e) -> { + }); + } + + /** + * Clears the future with the specified key from the cache remote value store, if caching is enabled + * and a remote store is set, so it will be re-fetched and stored on the next load request. + * + * @param key the key to remove + * @param handler a handler that will be called after the async remote clear completes + * + * @return the data loader for fluent coding + */ + public DataLoader clear(K key, BiConsumer handler) { Object cacheKey = getCacheKey(key); synchronized (this) { futureCache.delete(cacheKey); + valueCache.delete(key).whenComplete(handler); } return this; } @@ -641,14 +663,29 @@ public DataLoader clear(K key) { * @return the data loader for fluent coding */ public DataLoader clearAll() { + return clearAll((v, e) -> { + }); + } + + /** + * Clears the entire cache map of the loader, and of the cached value store. + * + * @param handler a handler that will be called after the async remote clear all completes + * + * @return the data loader for fluent coding + */ + public DataLoader clearAll(BiConsumer handler) { synchronized (this) { futureCache.clear(); + valueCache.clear().whenComplete(handler); } return this; } /** - * Primes the cache with the given key and value. + * Primes the cache with the given key and value. Note this will only prime the future cache + * and not the value store. Use {@link ValueCache#set(Object, Object)} if you want + * o prime it with values before use * * @param key the key * @param value the value diff --git a/src/main/java/org/dataloader/DataLoaderHelper.java b/src/main/java/org/dataloader/DataLoaderHelper.java index bd93814..fab4f9b 100644 --- a/src/main/java/org/dataloader/DataLoaderHelper.java +++ b/src/main/java/org/dataloader/DataLoaderHelper.java @@ -16,6 +16,7 @@ import java.util.concurrent.CompletableFuture; import java.util.concurrent.CompletionStage; import java.util.concurrent.atomic.AtomicReference; +import java.util.function.BiConsumer; import java.util.stream.Collectors; import static java.util.Collections.emptyList; @@ -61,17 +62,25 @@ Object getCallContext() { private final DataLoader dataLoader; private final Object batchLoadFunction; private final DataLoaderOptions loaderOptions; - private final CacheMap> futureCache; + private final CacheMap futureCache; + private final ValueCache valueCache; private final List>> loaderQueue; private final StatisticsCollector stats; private final Clock clock; private final AtomicReference lastDispatchTime; - DataLoaderHelper(DataLoader dataLoader, Object batchLoadFunction, DataLoaderOptions loaderOptions, CacheMap> futureCache, StatisticsCollector stats, Clock clock) { + DataLoaderHelper(DataLoader dataLoader, + Object batchLoadFunction, + DataLoaderOptions loaderOptions, + CacheMap futureCache, + ValueCache valueCache, + StatisticsCollector stats, + Clock clock) { this.dataLoader = dataLoader; this.batchLoadFunction = batchLoadFunction; this.loaderOptions = loaderOptions; this.futureCache = futureCache; + this.valueCache = valueCache; this.loaderQueue = new ArrayList<>(); this.stats = stats; this.clock = clock; @@ -120,35 +129,13 @@ CompletableFuture load(K key, Object loadContext) { boolean batchingEnabled = loaderOptions.batchingEnabled(); boolean cachingEnabled = loaderOptions.cachingEnabled(); - Object cacheKey = null; - if (cachingEnabled) { - if (loadContext == null) { - cacheKey = getCacheKey(key); - } else { - cacheKey = getCacheKeyWithContext(key, loadContext); - } - } stats.incrementLoadCount(); if (cachingEnabled) { - if (futureCache.containsKey(cacheKey)) { - stats.incrementCacheHitCount(); - return futureCache.get(cacheKey); - } - } - - CompletableFuture future = new CompletableFuture<>(); - if (batchingEnabled) { - loaderQueue.add(new LoaderQueueEntry<>(key, future, loadContext)); + return loadFromCache(key, loadContext, batchingEnabled); } else { - stats.incrementBatchLoadCountBy(1); - // immediate execution of batch function - future = invokeLoaderImmediately(key, loadContext); - } - if (cachingEnabled) { - futureCache.set(cacheKey, future); + return queueOrInvokeLoader(key, loadContext, batchingEnabled); } - return future; } } @@ -296,6 +283,66 @@ private void possiblyClearCacheEntriesOnExceptions(List keys) { } } + private CompletableFuture loadFromCache(K key, Object loadContext, boolean batchingEnabled) { + final Object cacheKey = loadContext == null ? getCacheKey(key) : getCacheKeyWithContext(key, loadContext); + + if (futureCache.containsKey(cacheKey)) { + // We already have a promise for this key, no need to check value cache or queue up load + stats.incrementCacheHitCount(); + return futureCache.get(cacheKey); + } + + /* + We haven't been asked for this key yet. We want to do one of two things: + + 1. Check if our cache store has it. If so: + a. Get the value from the cache store + b. Add a recovery case so we queue the load if fetching from cache store fails + c. Put that future in our futureCache to hit the early return next time + d. Return the resilient future + 2. If not in value cache: + a. queue or invoke the load + b. Add a success handler to store the result in the cache store + c. Return the result + */ + final CompletableFuture future = new CompletableFuture<>(); + + valueCache.get(cacheKey).whenComplete((cachedValue, getCallEx) -> { + if (getCallEx == null) { + future.complete(cachedValue); + } else { + queueOrInvokeLoader(key, loadContext, batchingEnabled) + .whenComplete(setValueIntoCacheAndCompleteFuture(cacheKey, future)); + } + }); + + futureCache.set(cacheKey, future); + + return future; + } + + private BiConsumer setValueIntoCacheAndCompleteFuture(Object cacheKey, CompletableFuture future) { + return (result, loadCallEx) -> { + if (loadCallEx == null) { + valueCache.set(cacheKey, result) + .whenComplete((v, setCallExIgnored) -> future.complete(result)); + } else { + future.completeExceptionally(loadCallEx); + } + }; + } + + private CompletableFuture queueOrInvokeLoader(K key, Object loadContext, boolean batchingEnabled) { + if (batchingEnabled) { + CompletableFuture future = new CompletableFuture<>(); + loaderQueue.add(new LoaderQueueEntry<>(key, future, loadContext)); + return future; + } else { + stats.incrementBatchLoadCountBy(1); + // immediate execution of batch function + return invokeLoaderImmediately(key, loadContext); + } + } CompletableFuture invokeLoaderImmediately(K key, Object keyContext) { List keys = singletonList(key); diff --git a/src/main/java/org/dataloader/DataLoaderOptions.java b/src/main/java/org/dataloader/DataLoaderOptions.java index ac54c3e..89530e1 100644 --- a/src/main/java/org/dataloader/DataLoaderOptions.java +++ b/src/main/java/org/dataloader/DataLoaderOptions.java @@ -38,8 +38,9 @@ public class DataLoaderOptions { private boolean batchingEnabled; private boolean cachingEnabled; private boolean cachingExceptionsEnabled; - private CacheKey cacheKeyFunction; - private CacheMap cacheMap; + private CacheKey cacheKeyFunction; + private CacheMap cacheMap; + private ValueCache valueCache; private int maxBatchSize; private Supplier statisticsCollector; private BatchLoaderContextProvider environmentProvider; @@ -166,7 +167,7 @@ public Optional cacheKeyFunction() { * * @return the data loader options for fluent coding */ - public DataLoaderOptions setCacheKeyFunction(CacheKey cacheKeyFunction) { + public DataLoaderOptions setCacheKeyFunction(CacheKey cacheKeyFunction) { this.cacheKeyFunction = cacheKeyFunction; return this; } @@ -178,7 +179,7 @@ public DataLoaderOptions setCacheKeyFunction(CacheKey cacheKeyFunction) { * * @return an optional with the cache map instance, or empty */ - public Optional cacheMap() { + public Optional> cacheMap() { return Optional.ofNullable(cacheMap); } @@ -189,7 +190,7 @@ public Optional cacheMap() { * * @return the data loader options for fluent coding */ - public DataLoaderOptions setCacheMap(CacheMap cacheMap) { + public DataLoaderOptions setCacheMap(CacheMap cacheMap) { this.cacheMap = cacheMap; return this; } @@ -256,4 +257,27 @@ public DataLoaderOptions setBatchLoaderContextProvider(BatchLoaderContextProvide this.environmentProvider = nonNull(contextProvider); return this; } + + /** + * Gets the (optional) cache store implementation that is used for value caching, if caching is enabled. + *

+ * If missing, a no-op implementation will be used. + * + * @return an optional with the cache store instance, or empty + */ + public Optional> valueCache() { + return Optional.ofNullable(valueCache); + } + + /** + * Sets the value cache implementation to use for caching values, if caching is enabled. + * + * @param valueCache the value cache instance + * + * @return the data loader options for fluent coding + */ + public DataLoaderOptions setValueCache(ValueCache valueCache) { + this.valueCache = valueCache; + return this; + } } diff --git a/src/main/java/org/dataloader/ValueCache.java b/src/main/java/org/dataloader/ValueCache.java new file mode 100644 index 0000000..31042c6 --- /dev/null +++ b/src/main/java/org/dataloader/ValueCache.java @@ -0,0 +1,87 @@ +package org.dataloader; + +import org.dataloader.annotations.PublicSpi; +import org.dataloader.impl.NoOpValueCache; + +import java.util.concurrent.CompletableFuture; + +/** + * The {@link ValueCache} is used by data loaders that use caching and want a long-lived or external cache + * of values. The {@link ValueCache} is used as a place to cache values when they come back from + *

+ * It differs from {@link CacheMap} which is in fact a cache of promises to values aka {@link CompletableFuture}<V> and it rather suited + * to be a wrapper of a long lived or external value cache. {@link CompletableFuture}s cant be easily placed in an external cache + * outside the JVM say, hence the need for the {@link ValueCache}. + *

+ * {@link DataLoader}s use a two stage cache strategy if caching is enabled. If the {@link CacheMap} already has the promise to a value + * that is used. If not then the {@link ValueCache} is asked for a value, if it has one then that is returned (and cached as a promise in the {@link CacheMap}. + * If there is no value then the key is queued and loaded via the {@link BatchLoader} calls. The returned values will then be stored in + * the {@link ValueCache} and the promises to those values are also stored in the {@link CacheMap}. + *

+ * The default implementation is a no-op store which replies with the key always missing and doesn't + * store any actual results. This is to avoid duplicating the stored data between the {@link CacheMap} + * out of the box. + *

+ * The API signature uses completable futures because the backing implementation MAY be a remote external cache + * and hence exceptions may happen in retrieving values. + * + * @param the type of cache keys + * @param the type of cache values + * + * @author Craig Day + */ +@PublicSpi +public interface ValueCache { + + + /** + * Creates a new value cache, using the default no-op implementation. + * + * @param the type of cache keys + * @param the type of cache values + * + * @return the cache store + */ + static ValueCache defaultValueCache() { + //noinspection unchecked + return (ValueCache) NoOpValueCache.NOOP; + } + + /** + * Gets the specified key from the store. if the key si not present, then the implementation MUST return an exceptionally completed future + * and not null because null is a valid cacheable value. Any exception is will cause {@link DataLoader} to load the key via batch loading + * instead. + * + * @param key the key to retrieve + * + * @return a future containing the cached value (which maybe null) or exceptionally completed future if the key does + * not exist in the cache. + */ + CompletableFuture get(K key); + + /** + * Stores the value with the specified key, or updates it if the key already exists. + * + * @param key the key to store + * @param value the value to store + * + * @return a future containing the stored value for fluent composition + */ + CompletableFuture set(K key, V value); + + /** + * Deletes the entry with the specified key from the store, if it exists. + * + * @param key the key to delete + * + * @return a void future for error handling and fluent composition + */ + CompletableFuture delete(K key); + + /** + * Clears all entries from the store. + * + * @return a void future for error handling and fluent composition + */ + CompletableFuture clear(); +} \ No newline at end of file diff --git a/src/main/java/org/dataloader/impl/DefaultCacheMap.java b/src/main/java/org/dataloader/impl/DefaultCacheMap.java index 7245ce2..9346ad8 100644 --- a/src/main/java/org/dataloader/impl/DefaultCacheMap.java +++ b/src/main/java/org/dataloader/impl/DefaultCacheMap.java @@ -21,19 +21,20 @@ import java.util.HashMap; import java.util.Map; +import java.util.concurrent.CompletableFuture; /** - * Default implementation of {@link CacheMap} that is based on a regular {@link java.util.LinkedHashMap}. + * Default implementation of {@link CacheMap} that is based on a regular {@link java.util.HashMap}. * - * @param type parameter indicating the type of the cache keys + * @param type parameter indicating the type of the cache keys * @param type parameter indicating the type of the data that is cached * * @author Arnold Schrijver */ @Internal -public class DefaultCacheMap implements CacheMap { +public class DefaultCacheMap implements CacheMap { - private final Map cache; + private final Map> cache; /** * Default constructor @@ -46,15 +47,16 @@ public DefaultCacheMap() { * {@inheritDoc} */ @Override - public boolean containsKey(U key) { + public boolean containsKey(K key) { return cache.containsKey(key); } + /** * {@inheritDoc} */ @Override - public V get(U key) { + public CompletableFuture get(K key) { return cache.get(key); } @@ -62,7 +64,7 @@ public V get(U key) { * {@inheritDoc} */ @Override - public CacheMap set(U key, V value) { + public CacheMap set(K key, CompletableFuture value) { cache.put(key, value); return this; } @@ -71,7 +73,7 @@ public CacheMap set(U key, V value) { * {@inheritDoc} */ @Override - public CacheMap delete(U key) { + public CacheMap delete(K key) { cache.remove(key); return this; } @@ -80,7 +82,7 @@ public CacheMap delete(U key) { * {@inheritDoc} */ @Override - public CacheMap clear() { + public CacheMap clear() { cache.clear(); return this; } diff --git a/src/main/java/org/dataloader/impl/NoOpValueCache.java b/src/main/java/org/dataloader/impl/NoOpValueCache.java new file mode 100644 index 0000000..bd82f03 --- /dev/null +++ b/src/main/java/org/dataloader/impl/NoOpValueCache.java @@ -0,0 +1,56 @@ +package org.dataloader.impl; + + +import org.dataloader.ValueCache; +import org.dataloader.annotations.Internal; + +import java.util.concurrent.CompletableFuture; + +/** + * Implementation of {@link ValueCache} that does nothing. + *

+ * We don't want to store values in memory twice, so when using the default store we just + * say we never have the key and complete the other methods by doing nothing. + * + * @param the type of cache keys + * @param the type of cache values + * + * @author Craig Day + */ +@Internal +public class NoOpValueCache implements ValueCache { + + public static NoOpValueCache NOOP = new NoOpValueCache<>(); + + /** + * {@inheritDoc} + */ + @Override + public CompletableFuture get(K key) { + return CompletableFutureKit.failedFuture(new UnsupportedOperationException()); + } + + /** + * {@inheritDoc} + */ + @Override + public CompletableFuture set(K key, V value) { + return CompletableFuture.completedFuture(value); + } + + /** + * {@inheritDoc} + */ + @Override + public CompletableFuture delete(K key) { + return CompletableFuture.completedFuture(null); + } + + /** + * {@inheritDoc} + */ + @Override + public CompletableFuture clear() { + return CompletableFuture.completedFuture(null); + } +} \ No newline at end of file diff --git a/src/test/java/ReadmeExamples.java b/src/test/java/ReadmeExamples.java index ccdd555..6cef375 100644 --- a/src/test/java/ReadmeExamples.java +++ b/src/test/java/ReadmeExamples.java @@ -214,12 +214,12 @@ public boolean containsKey(Object key) { } @Override - public Object get(Object key) { + public CompletableFuture get(Object key) { return null; } @Override - public CacheMap set(Object key, Object value) { + public CacheMap set(Object key, CompletableFuture value) { return null; } diff --git a/src/test/java/org/dataloader/DataLoaderTest.java b/src/test/java/org/dataloader/DataLoaderTest.java index 18aa1ba..08d5b44 100644 --- a/src/test/java/org/dataloader/DataLoaderTest.java +++ b/src/test/java/org/dataloader/DataLoaderTest.java @@ -16,6 +16,7 @@ package org.dataloader; +import org.dataloader.fixtures.CustomCacheMap; import org.dataloader.fixtures.JsonObject; import org.dataloader.fixtures.TestKit; import org.dataloader.fixtures.User; diff --git a/src/test/java/org/dataloader/DataLoaderValueCacheTest.java b/src/test/java/org/dataloader/DataLoaderValueCacheTest.java new file mode 100644 index 0000000..1c54e91 --- /dev/null +++ b/src/test/java/org/dataloader/DataLoaderValueCacheTest.java @@ -0,0 +1,204 @@ +package org.dataloader; + +import com.github.benmanes.caffeine.cache.Cache; +import com.github.benmanes.caffeine.cache.Caffeine; +import org.dataloader.fixtures.CaffeineValueCache; +import org.dataloader.fixtures.CustomValueCache; +import org.junit.Test; + +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.TimeUnit; + +import static java.util.Arrays.asList; +import static java.util.Collections.emptyList; +import static java.util.Collections.singletonList; +import static org.awaitility.Awaitility.await; +import static org.dataloader.DataLoaderOptions.newOptions; +import static org.dataloader.fixtures.TestKit.idLoader; +import static org.dataloader.impl.CompletableFutureKit.failedFuture; +import static org.hamcrest.Matchers.equalTo; +import static org.junit.Assert.assertArrayEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertThat; +import static org.junit.Assert.assertTrue; + +public class DataLoaderValueCacheTest { + + @Test + public void test_by_default_we_have_no_value_caching() { + List> loadCalls = new ArrayList<>(); + DataLoaderOptions options = newOptions(); + DataLoader identityLoader = idLoader(options, loadCalls); + + CompletableFuture fA = identityLoader.load("a"); + CompletableFuture fB = identityLoader.load("b"); + + assertFalse(fA.isDone()); + assertFalse(fB.isDone()); + + await().until(identityLoader.dispatch()::isDone); + assertThat(fA.join(), equalTo("a")); + assertThat(fB.join(), equalTo("b")); + + assertThat(loadCalls, equalTo(singletonList(asList("a", "b")))); + + // futures are still cached but not values + + fA = identityLoader.load("a"); + fB = identityLoader.load("b"); + + assertTrue(fA.isDone()); + assertTrue(fB.isDone()); + + await().until(identityLoader.dispatch()::isDone); + assertThat(fA.join(), equalTo("a")); + assertThat(fB.join(), equalTo("b")); + + assertThat(loadCalls, equalTo(singletonList(asList("a", "b")))); + + } + + @Test + public void should_accept_a_remote_value_store_for_caching() { + CustomValueCache customStore = new CustomValueCache(); + List> loadCalls = new ArrayList<>(); + DataLoaderOptions options = newOptions().setValueCache(customStore); + DataLoader identityLoader = idLoader(options, loadCalls); + + // Fetches as expected + + CompletableFuture fA = identityLoader.load("a"); + CompletableFuture fB = identityLoader.load("b"); + + await().until(identityLoader.dispatch()::isDone); + assertThat(fA.join(), equalTo("a")); + assertThat(fB.join(), equalTo("b")); + + assertThat(loadCalls, equalTo(singletonList(asList("a", "b")))); + assertArrayEquals(customStore.store.keySet().toArray(), asList("a", "b").toArray()); + + CompletableFuture future3 = identityLoader.load("c"); + CompletableFuture future2a = identityLoader.load("b"); + + await().until(identityLoader.dispatch()::isDone); + assertThat(future3.join(), equalTo("c")); + assertThat(future2a.join(), equalTo("b")); + + assertThat(loadCalls, equalTo(asList(asList("a", "b"), singletonList("c")))); + assertArrayEquals(customStore.store.keySet().toArray(), asList("a", "b", "c").toArray()); + + // Supports clear + + CompletableFuture fC = new CompletableFuture<>(); + identityLoader.clear("b", (v, e) -> fC.complete(v)); + await().until(fC::isDone); + assertArrayEquals(customStore.store.keySet().toArray(), asList("a", "c").toArray()); + + // Supports clear all + + CompletableFuture fCa = new CompletableFuture<>(); + identityLoader.clearAll((v, e) -> fCa.complete(v)); + await().until(fCa::isDone); + assertArrayEquals(customStore.store.keySet().toArray(), emptyList().toArray()); + } + + @Test + public void can_use_caffeine_for_caching() { + // + // Mostly to prove that some other CACHE library could be used + // as the backing value cache. Not really Caffeine specific. + // + Cache caffeineCache = Caffeine.newBuilder() + .expireAfterWrite(10, TimeUnit.MINUTES) + .maximumSize(100) + .build(); + + ValueCache customStore = new CaffeineValueCache(caffeineCache); + + List> loadCalls = new ArrayList<>(); + DataLoaderOptions options = newOptions().setValueCache(customStore); + DataLoader identityLoader = idLoader(options, loadCalls); + + // Fetches as expected + + CompletableFuture fA = identityLoader.load("a"); + CompletableFuture fB = identityLoader.load("b"); + + await().until(identityLoader.dispatch()::isDone); + assertThat(fA.join(), equalTo("a")); + assertThat(fB.join(), equalTo("b")); + + assertThat(loadCalls, equalTo(singletonList(asList("a", "b")))); + assertArrayEquals(caffeineCache.asMap().keySet().toArray(), asList("a", "b").toArray()); + + CompletableFuture fC = identityLoader.load("c"); + CompletableFuture fBa = identityLoader.load("b"); + + await().until(identityLoader.dispatch()::isDone); + assertThat(fC.join(), equalTo("c")); + assertThat(fBa.join(), equalTo("b")); + + assertThat(loadCalls, equalTo(asList(asList("a", "b"), singletonList("c")))); + assertArrayEquals(caffeineCache.asMap().keySet().toArray(), asList("a", "b", "c").toArray()); + } + + @Test + public void will_invoke_loader_if_CACHE_GET_call_throws_exception() { + CustomValueCache customStore = new CustomValueCache() { + + @Override + public CompletableFuture get(String key) { + if (key.equals("a")) { + return failedFuture(new IllegalStateException("no A")); + } + return super.get(key); + } + }; + customStore.set("a", "Not From Cache"); + customStore.set("b", "From Cache"); + + List> loadCalls = new ArrayList<>(); + DataLoaderOptions options = newOptions().setValueCache(customStore); + DataLoader identityLoader = idLoader(options, loadCalls); + + CompletableFuture fA = identityLoader.load("a"); + CompletableFuture fB = identityLoader.load("b"); + + await().until(identityLoader.dispatch()::isDone); + assertThat(fA.join(), equalTo("a")); + assertThat(fB.join(), equalTo("From Cache")); + + // a was not in cache (according to get) and hence needed to be loaded + assertThat(loadCalls, equalTo(singletonList(singletonList("a")))); + } + + @Test + public void will_still_work_if_CACHE_SET_call_throws_exception() { + CustomValueCache customStore = new CustomValueCache() { + @Override + public CompletableFuture set(String key, Object value) { + if (key.equals("a")) { + return failedFuture(new IllegalStateException("no A")); + } + return super.set(key, value); + } + }; + + List> loadCalls = new ArrayList<>(); + DataLoaderOptions options = newOptions().setValueCache(customStore); + DataLoader identityLoader = idLoader(options, loadCalls); + + CompletableFuture fA = identityLoader.load("a"); + CompletableFuture fB = identityLoader.load("b"); + + await().until(identityLoader.dispatch()::isDone); + assertThat(fA.join(), equalTo("a")); + assertThat(fB.join(), equalTo("b")); + + // a was not in cache (according to get) and hence needed to be loaded + assertThat(loadCalls, equalTo(singletonList(asList("a", "b")))); + assertArrayEquals(customStore.store.keySet().toArray(), singletonList("b").toArray()); + } +} diff --git a/src/test/java/org/dataloader/fixtures/CaffeineValueCache.java b/src/test/java/org/dataloader/fixtures/CaffeineValueCache.java new file mode 100644 index 0000000..2dce1a0 --- /dev/null +++ b/src/test/java/org/dataloader/fixtures/CaffeineValueCache.java @@ -0,0 +1,45 @@ +package org.dataloader.fixtures; + + +import com.github.benmanes.caffeine.cache.Cache; +import org.dataloader.ValueCache; +import org.dataloader.impl.CompletableFutureKit; + +import java.util.concurrent.CompletableFuture; + +public class CaffeineValueCache implements ValueCache { + + public final Cache cache; + + public CaffeineValueCache(Cache cache) { + this.cache = cache; + } + + @Override + public CompletableFuture get(String key) { + Object value = cache.getIfPresent(key); + if (value == null) { + // we use get exceptions here to indicate not in cache + return CompletableFutureKit.failedFuture(new RuntimeException(key + " not present")); + } + return CompletableFuture.completedFuture(value); + } + + @Override + public CompletableFuture set(String key, Object value) { + cache.put(key, value); + return CompletableFuture.completedFuture(value); + } + + @Override + public CompletableFuture delete(String key) { + cache.invalidate(key); + return CompletableFuture.completedFuture(null); + } + + @Override + public CompletableFuture clear() { + cache.invalidateAll(); + return CompletableFuture.completedFuture(null); + } +} \ No newline at end of file diff --git a/src/test/java/org/dataloader/CustomCacheMap.java b/src/test/java/org/dataloader/fixtures/CustomCacheMap.java similarity index 68% rename from src/test/java/org/dataloader/CustomCacheMap.java rename to src/test/java/org/dataloader/fixtures/CustomCacheMap.java index 505148d..51e6687 100644 --- a/src/test/java/org/dataloader/CustomCacheMap.java +++ b/src/test/java/org/dataloader/fixtures/CustomCacheMap.java @@ -1,11 +1,14 @@ -package org.dataloader; +package org.dataloader.fixtures; + +import org.dataloader.CacheMap; import java.util.LinkedHashMap; import java.util.Map; +import java.util.concurrent.CompletableFuture; public class CustomCacheMap implements CacheMap { - public Map stash; + public Map> stash; public CustomCacheMap() { stash = new LinkedHashMap<>(); @@ -17,12 +20,12 @@ public boolean containsKey(String key) { } @Override - public Object get(String key) { + public CompletableFuture get(String key) { return stash.get(key); } @Override - public CacheMap set(String key, Object value) { + public CacheMap set(String key, CompletableFuture value) { stash.put(key, value); return this; } diff --git a/src/test/java/org/dataloader/fixtures/CustomValueCache.java b/src/test/java/org/dataloader/fixtures/CustomValueCache.java new file mode 100644 index 0000000..d707175 --- /dev/null +++ b/src/test/java/org/dataloader/fixtures/CustomValueCache.java @@ -0,0 +1,40 @@ +package org.dataloader.fixtures; + + +import org.dataloader.ValueCache; +import org.dataloader.impl.CompletableFutureKit; + +import java.util.Map; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.ConcurrentHashMap; + +public class CustomValueCache implements ValueCache { + + public final Map store = new ConcurrentHashMap<>(); + + @Override + public CompletableFuture get(String key) { + if (!store.containsKey(key)) { + return CompletableFutureKit.failedFuture(new RuntimeException("The key is missing")); + } + return CompletableFuture.completedFuture(store.get(key)); + } + + @Override + public CompletableFuture set(String key, Object value) { + store.put(key, value); + return CompletableFuture.completedFuture(value); + } + + @Override + public CompletableFuture delete(String key) { + store.remove(key); + return CompletableFuture.completedFuture(null); + } + + @Override + public CompletableFuture clear() { + store.clear(); + return CompletableFuture.completedFuture(null); + } +} \ No newline at end of file