Skip to content

docs: bounded cache link fixes #1816

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Mar 14, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@
import com.github.benmanes.caffeine.cache.Cache;

/**
* Caffein cache wrapper to be used in a {@link BoundedItemStore}
* Caffeine cache wrapper to be used in a {@link BoundedItemStore}
*/
public class CaffeineBoundedCache<K, R> implements BoundedCache<K, R> {

private Cache<K, R> cache;
private final Cache<K, R> cache;

public CaffeineBoundedCache(Cache<K, R> cache) {
this.cache = cache;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,22 @@
import com.github.benmanes.caffeine.cache.Caffeine;

/**
* The idea about CaffeinBoundedItemStore-s is that, caffeine will cache the resources which were
* recently used, and will evict resource, which are not used for a while. This is ideal from the
* perspective that on startup controllers reconcile all resources (this is why a maxSize not ideal)
* but after a while it can happen (well depending on the controller and domain) that only some
* resources are actually active, thus related events happen. So in case large amount of custom
* resources only the active once will remain in the cache. Note that if a resource is reconciled
* all the secondary resources are usually reconciled too, in that case all those resources are
* fetched and populated to the cache, and will remain there for some time, for a subsequent
* reconciliations.
* A factory for <a href="https://github.com/ben-manes/caffeine">Caffeine</a>-backed
* {@link BoundedItemStore}. The implementation uses a {@link CaffeineBoundedCache} to store
* resources and progressively evict them if they haven't been used in a while. The idea about
* CaffeinBoundedItemStore-s is that, caffeine will cache the resources which were recently used,
* and will evict resource, which are not used for a while. This is ideal for startup performance
* and efficiency when all resources should be cached to avoid undue load on the API server. This is
* why setting a maximal cache size is not practical and the approach of evicting least recently
* used resources was chosen. However, depending on controller implementations and domains, it could
* happen that some / many of these resources are then seldom or even reconciled anymore. In that
* situation, large amounts of memory might be consumed to cache resources that are never used
* again.
* <p>
* Note that if a resource is reconciled and is not present anymore in cache, it will transparently
* be fetched again from the API server. Similarly, since associated secondary resources are usually
* reconciled too, they might need to be fetched and populated to the cache, and will remain there
* for some time, for subsequent reconciliations.
*/
public class CaffeineBoundedItemStores {

Expand All @@ -30,6 +37,7 @@ private CaffeineBoundedItemStores() {}
* @return the ItemStore implementation
* @param <R> resource type
*/
@SuppressWarnings("unused")
public static <R extends HasMetadata> BoundedItemStore<R> boundedItemStore(
KubernetesClient client, Class<R> rClass,
Duration accessExpireDuration) {
Expand Down
46 changes: 26 additions & 20 deletions docs/documentation/features.md
Original file line number Diff line number Diff line change
Expand Up @@ -722,9 +722,9 @@ to add the following dependencies to your project:
```xml

<dependency>
<groupId>io.fabric8</groupId>
<artifactId>crd-generator-apt</artifactId>
<scope>provided</scope>
<groupId>io.fabric8</groupId>
<artifactId>crd-generator-apt</artifactId>
<scope>provided</scope>
</dependency>
```

Expand All @@ -742,26 +742,32 @@ with a `mycrs` plural form will result in 2 files:

## Optimizing Caches

One of the ideas around operator pattern, is that all the relevant resources are cached, thus reconciliation is usually
very fast (especially if it does not need to update resources) since it's mostly working with in memory state.
However or large clusters, caching huge amount of primary and secondary resources might consume lots of memory.
There are some semi-experimental (experimental in terms that it works, but we need feedback from real production usage)
features to optimize memory usage of controllers.
One of the ideas around the operator pattern is that all the relevant resources are cached, thus reconciliation is
usually very fast (especially if no resources are updated in the process) since the operator is then mostly working with
in-memory state. However for large clusters, caching huge amount of primary and secondary resources might consume lots
of memory. JOSDK provides ways to mitigate this issue and optimize the memory usage of controllers. While these features
are working and tested, we need feedback from real production usage.

### Bounded Caches for Informers
### Bounded Caches for Informers

Limiting caches for informers - thus for Kubernetes resources, both controllers primary resource - is supported for now.
The idea with the implementation that is provided, is that resources are in the cache for a limited time.
So for use cases, when a resource is only frequently reconciled when it is created, and later no or
occasionally reconciled, will be evicted from the cache, since the resources are not accessed.
If a resource accessed in the future but not in the cache, the bounded cache implementation will fetch it from
the service if needed.
Note that on start of a controller all the resources are reconciled, for this reason explicitly setting a maximal
size of a cache might not be ideal. In other words it is desired to have all the resources in the cache at startup,
but not later if not accessed.
Limiting caches for informers - thus for Kubernetes resources - is supported by ensuring that resources are in the cache
for a limited time, via a cache eviction of least recently used resources. This means that when resources are created
and frequently reconciled, they stay "hot" in the cache. However, if, over time, a given resource "cools" down, i.e. it
becomes less and less used to the point that it might not be reconciled anymore, it will eventually get evicted from the
cache to free up memory. If such an evicted resource were to become reconciled again, the bounded cache implementation
would then fetch it from the API server and the "hot/cold" cycle would start anew.

See usage of the related implementation using Caffein cache in integration tests for [primary resource](https://github.com/java-operator-sdk/java-operator-sdk/blob/10e11e587447667ef0da1ddb29e0ba15fcd24ada/caffein-bounded-cache-support/src/test/java/io/javaoperatorsdk/operator/processing/event/source/cache/CaffeinBoundedCacheNamespacedIT.java#L19-L19) and for an [informer](https://github.com/java-operator-sdk/java-operator-sdk/blob/10e11e587447667ef0da1ddb29e0ba15fcd24ada/caffein-bounded-cache-support/src/test/java/io/javaoperatorsdk/operator/processing/event/source/cache/sample/AbstractTestReconciler.java#L84-L93).
Since all resources need to be reconciled when a controller start, it is not practical to set a maximal cache size as
it's desirable that all resources be cached as soon as possible to make the initial reconciliation process on start as
fast and efficient as possible, avoiding undue load on the API server. It's therefore more interesting to gradually
evict cold resources than try to limit cache sizes.

See also [CaffeinBoundedItemStores](https://github.com/java-operator-sdk/java-operator-sdk/blob/10e11e587447667ef0da1ddb29e0ba15fcd24ada/caffein-bounded-cache-support/src/main/java/io/javaoperatorsdk/operator/processing/event/source/cache/CaffeinBoundedItemStores.java#L22-L22)
See usage of the related implementation using [Caffeine](https://github.com/ben-manes/caffeine) cache in integration
tests
for [primary resources](https://github.com/java-operator-sdk/java-operator-sdk/blob/902c8a562dfd7f8993a52e03473a7ad4b00f378b/caffeine-bounded-cache-support/src/test/java/io/javaoperatorsdk/operator/processing/event/source/cache/sample/AbstractTestReconciler.java#L29-L29).

See
also [CaffeineBoundedItemStores](https://github.com/java-operator-sdk/java-operator-sdk/blob/902c8a562dfd7f8993a52e03473a7ad4b00f378b/caffeine-bounded-cache-support/src/main/java/io/javaoperatorsdk/operator/processing/event/source/cache/CaffeineBoundedItemStores.java)
for more details.