Skip to content

Commit caf13ce

Browse files
csvirimetacosm
andauthored
docs: bounded cache link fixes (#1816)
Co-authored-by: Chris Laprun <metacosm@gmail.com>
1 parent 902c8a5 commit caf13ce

File tree

3 files changed

+45
-31
lines changed

3 files changed

+45
-31
lines changed

caffeine-bounded-cache-support/src/main/java/io/javaoperatorsdk/operator/processing/event/source/cache/CaffeineBoundedCache.java

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,11 +3,11 @@
33
import com.github.benmanes.caffeine.cache.Cache;
44

55
/**
6-
* Caffein cache wrapper to be used in a {@link BoundedItemStore}
6+
* Caffeine cache wrapper to be used in a {@link BoundedItemStore}
77
*/
88
public class CaffeineBoundedCache<K, R> implements BoundedCache<K, R> {
99

10-
private Cache<K, R> cache;
10+
private final Cache<K, R> cache;
1111

1212
public CaffeineBoundedCache(Cache<K, R> cache) {
1313
this.cache = cache;

caffeine-bounded-cache-support/src/main/java/io/javaoperatorsdk/operator/processing/event/source/cache/CaffeineBoundedItemStores.java

Lines changed: 17 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -9,15 +9,22 @@
99
import com.github.benmanes.caffeine.cache.Caffeine;
1010

1111
/**
12-
* The idea about CaffeinBoundedItemStore-s is that, caffeine will cache the resources which were
13-
* recently used, and will evict resource, which are not used for a while. This is ideal from the
14-
* perspective that on startup controllers reconcile all resources (this is why a maxSize not ideal)
15-
* but after a while it can happen (well depending on the controller and domain) that only some
16-
* resources are actually active, thus related events happen. So in case large amount of custom
17-
* resources only the active once will remain in the cache. Note that if a resource is reconciled
18-
* all the secondary resources are usually reconciled too, in that case all those resources are
19-
* fetched and populated to the cache, and will remain there for some time, for a subsequent
20-
* reconciliations.
12+
* A factory for <a href="https://github.com/ben-manes/caffeine">Caffeine</a>-backed
13+
* {@link BoundedItemStore}. The implementation uses a {@link CaffeineBoundedCache} to store
14+
* resources and progressively evict them if they haven't been used in a while. The idea about
15+
* CaffeinBoundedItemStore-s is that, caffeine will cache the resources which were recently used,
16+
* and will evict resource, which are not used for a while. This is ideal for startup performance
17+
* and efficiency when all resources should be cached to avoid undue load on the API server. This is
18+
* why setting a maximal cache size is not practical and the approach of evicting least recently
19+
* used resources was chosen. However, depending on controller implementations and domains, it could
20+
* happen that some / many of these resources are then seldom or even reconciled anymore. In that
21+
* situation, large amounts of memory might be consumed to cache resources that are never used
22+
* again.
23+
* <p>
24+
* Note that if a resource is reconciled and is not present anymore in cache, it will transparently
25+
* be fetched again from the API server. Similarly, since associated secondary resources are usually
26+
* reconciled too, they might need to be fetched and populated to the cache, and will remain there
27+
* for some time, for subsequent reconciliations.
2128
*/
2229
public class CaffeineBoundedItemStores {
2330

@@ -30,6 +37,7 @@ private CaffeineBoundedItemStores() {}
3037
* @return the ItemStore implementation
3138
* @param <R> resource type
3239
*/
40+
@SuppressWarnings("unused")
3341
public static <R extends HasMetadata> BoundedItemStore<R> boundedItemStore(
3442
KubernetesClient client, Class<R> rClass,
3543
Duration accessExpireDuration) {

docs/documentation/features.md

Lines changed: 26 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -722,9 +722,9 @@ to add the following dependencies to your project:
722722
```xml
723723

724724
<dependency>
725-
<groupId>io.fabric8</groupId>
726-
<artifactId>crd-generator-apt</artifactId>
727-
<scope>provided</scope>
725+
<groupId>io.fabric8</groupId>
726+
<artifactId>crd-generator-apt</artifactId>
727+
<scope>provided</scope>
728728
</dependency>
729729
```
730730

@@ -742,26 +742,32 @@ with a `mycrs` plural form will result in 2 files:
742742

743743
## Optimizing Caches
744744

745-
One of the ideas around operator pattern, is that all the relevant resources are cached, thus reconciliation is usually
746-
very fast (especially if it does not need to update resources) since it's mostly working with in memory state.
747-
However or large clusters, caching huge amount of primary and secondary resources might consume lots of memory.
748-
There are some semi-experimental (experimental in terms that it works, but we need feedback from real production usage)
749-
features to optimize memory usage of controllers.
745+
One of the ideas around the operator pattern is that all the relevant resources are cached, thus reconciliation is
746+
usually very fast (especially if no resources are updated in the process) since the operator is then mostly working with
747+
in-memory state. However for large clusters, caching huge amount of primary and secondary resources might consume lots
748+
of memory. JOSDK provides ways to mitigate this issue and optimize the memory usage of controllers. While these features
749+
are working and tested, we need feedback from real production usage.
750750

751-
### Bounded Caches for Informers
751+
### Bounded Caches for Informers
752752

753-
Limiting caches for informers - thus for Kubernetes resources, both controllers primary resource - is supported for now.
754-
The idea with the implementation that is provided, is that resources are in the cache for a limited time.
755-
So for use cases, when a resource is only frequently reconciled when it is created, and later no or
756-
occasionally reconciled, will be evicted from the cache, since the resources are not accessed.
757-
If a resource accessed in the future but not in the cache, the bounded cache implementation will fetch it from
758-
the service if needed.
759-
Note that on start of a controller all the resources are reconciled, for this reason explicitly setting a maximal
760-
size of a cache might not be ideal. In other words it is desired to have all the resources in the cache at startup,
761-
but not later if not accessed.
753+
Limiting caches for informers - thus for Kubernetes resources - is supported by ensuring that resources are in the cache
754+
for a limited time, via a cache eviction of least recently used resources. This means that when resources are created
755+
and frequently reconciled, they stay "hot" in the cache. However, if, over time, a given resource "cools" down, i.e. it
756+
becomes less and less used to the point that it might not be reconciled anymore, it will eventually get evicted from the
757+
cache to free up memory. If such an evicted resource were to become reconciled again, the bounded cache implementation
758+
would then fetch it from the API server and the "hot/cold" cycle would start anew.
762759

763-
See usage of the related implementation using Caffein cache in integration tests for [primary resource](https://github.com/java-operator-sdk/java-operator-sdk/blob/10e11e587447667ef0da1ddb29e0ba15fcd24ada/caffein-bounded-cache-support/src/test/java/io/javaoperatorsdk/operator/processing/event/source/cache/CaffeinBoundedCacheNamespacedIT.java#L19-L19) and for an [informer](https://github.com/java-operator-sdk/java-operator-sdk/blob/10e11e587447667ef0da1ddb29e0ba15fcd24ada/caffein-bounded-cache-support/src/test/java/io/javaoperatorsdk/operator/processing/event/source/cache/sample/AbstractTestReconciler.java#L84-L93).
760+
Since all resources need to be reconciled when a controller start, it is not practical to set a maximal cache size as
761+
it's desirable that all resources be cached as soon as possible to make the initial reconciliation process on start as
762+
fast and efficient as possible, avoiding undue load on the API server. It's therefore more interesting to gradually
763+
evict cold resources than try to limit cache sizes.
764764

765-
See also [CaffeinBoundedItemStores](https://github.com/java-operator-sdk/java-operator-sdk/blob/10e11e587447667ef0da1ddb29e0ba15fcd24ada/caffein-bounded-cache-support/src/main/java/io/javaoperatorsdk/operator/processing/event/source/cache/CaffeinBoundedItemStores.java#L22-L22)
765+
See usage of the related implementation using [Caffeine](https://github.com/ben-manes/caffeine) cache in integration
766+
tests
767+
for [primary resources](https://github.com/java-operator-sdk/java-operator-sdk/blob/902c8a562dfd7f8993a52e03473a7ad4b00f378b/caffeine-bounded-cache-support/src/test/java/io/javaoperatorsdk/operator/processing/event/source/cache/sample/AbstractTestReconciler.java#L29-L29).
768+
769+
See
770+
also [CaffeineBoundedItemStores](https://github.com/java-operator-sdk/java-operator-sdk/blob/902c8a562dfd7f8993a52e03473a7ad4b00f378b/caffeine-bounded-cache-support/src/main/java/io/javaoperatorsdk/operator/processing/event/source/cache/CaffeineBoundedItemStores.java)
771+
for more details.
766772

767773

0 commit comments

Comments
 (0)