feat: primary resource caching for followup reconciliation(s)

docs/content/en/docs/documentation/reconciler.md

@@ -169,3 +169,117 @@ You can specify the name of the finalizer to use for your `Reconciler` using the
 annotation. If you do not specify a finalizer name, one will be automatically generated for you.
 
 From v5, by default, the finalizer is added using Server Side Apply. See also `UpdateControl` in docs.
+
+### Making sure the primary resource is up to date for the next reconciliation
+
+It is typical to want to update the status subresource with the information that is available during the reconciliation.
+This is sometimes referred to as the last observed state. When the primary resource is updated, though, the framework
+does not cache the resource directly, relying instead on the propagation of the update to the underlying informer's
+cache. It can, therefore, happen that, if other events trigger other reconciliations before the informer cache gets
+updated, your reconciler does not see the latest version of the primary resource. While this might not typically be a
+problem in most cases, as caches eventually become consistent, depending on your reconciliation logic, you might still
+require the latest status version possible, for example if the status subresource is used as a communication mechanism,
+see [Representing Allocated Values](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#representing-allocated-values)
+from the Kubernetes docs for more details.
+
+The framework provides utilities to help with these use cases with 
+[`PrimaryUpdateAndCacheUtils`](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/PrimaryUpdateAndCacheUtils.java).
+These utility methods come in two flavors:
+
+#### Using internal cache
+
+In almost all cases for this purpose, you can use internal caches:
+
+```java
+ @Override
+public UpdateControl<StatusPatchCacheCustomResource> reconcile(
+        StatusPatchCacheCustomResource resource, Context<StatusPatchCacheCustomResource> context) {
+
+    // omitted logic
+
+    // update with SSA requires a fresh copy
+    var freshCopy = createFreshCopy(primary);
+    freshCopy.getStatus().setValue(statusWithState());
+
+    var updatedResource = PrimaryUpdateAndCacheUtils.ssaPatchAndCacheStatus(resource, freshCopy, context);
+
+    return UpdateControl.noUpdate();
+  }
+```
+
+In the background `PrimaryUpdateAndCacheUtils.ssaPatchAndCacheStatus` puts the result of the update into an internal
+cache and will make sure that the next reconciliation will contain the most recent version of the resource. Note that it
+is not necessarily the version of the resource you got as response from the update, it can be newer since other parties
+can do additional updates meanwhile, but if not explicitly modified, it will contain the up-to-date status.
+
+See related integration test [here](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework/src/test/java/io/javaoperatorsdk/operator/baseapi/statuscache/internal).
+
+This approach works with the default configuration of the framework and should be good to go in most of the cases.
+Without going further into the details, this won't work if `ConfigurationService.parseResourceVersionsForEventFilteringAndCaching`
+is set to `false` (more precisely there are some edge cases when it won't work). For that case framework provides the following solution:
+
+#### Fallback approach: using `PrimaryResourceCache` cache
+
+As an alternative, for very rare cases when `ConfigurationService.parseResourceVersionsForEventFilteringAndCaching` 
+needs to be set to `false` you can use an explicit caching approach:
+
+```java
+
+// We on purpose don't use the provided predicate to show what a custom one could look like.
+  private final PrimaryResourceCache<StatusPatchPrimaryCacheCustomResource> cache =
+      new PrimaryResourceCache<>(
+          (statusPatchCacheCustomResourcePair, statusPatchCacheCustomResource) ->
+              statusPatchCacheCustomResource.getStatus().getValue()
+                  >= statusPatchCacheCustomResourcePair.afterUpdate().getStatus().getValue());
+
+  @Override
+  public UpdateControl<StatusPatchPrimaryCacheCustomResource> reconcile(
+          StatusPatchPrimaryCacheCustomResource primary,
+          Context<StatusPatchPrimaryCacheCustomResource> context) {
+
+    // cache will compare the current and the cached resource and return the more recent. (And evict the old)   
+    primary = cache.getFreshResource(primary);
+
+    // omitted logic  
+
+    var freshCopy = createFreshCopy(primary);
+
+    freshCopy.getStatus().setValue(statusWithState());
+
+    var updated =
+            PrimaryUpdateAndCacheUtils.ssaPatchAndCacheStatus(primary, freshCopy, context, cache);
+
+    return UpdateControl.noUpdate();
+  }
+
+  @Override
+  public DeleteControl cleanup(
+          StatusPatchPrimaryCacheCustomResource resource,
+          Context<StatusPatchPrimaryCacheCustomResource> context)
+          throws Exception {
+    // cleanup the cache on resource deletion  
+    cache.cleanup(resource);
+    return DeleteControl.defaultDelete();
+  }
+
+```
+
+[`PrimaryResourceCache`](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/support/PrimaryResourceCache.java)
+is designed for this purpose. As shown in the example above, it is up to you to provide a predicate to determine if the
+resource is more recent than the one available. In other words, when to evict the resource from the cache. Typically, as
+shown in
+the [integration test](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework/src/test/java/io/javaoperatorsdk/operator/baseapi/statuscache/primarycache)
+you can have a counter in status to check on that.
+
+Since all of this happens explicitly, you cannot use this approach for managed dependent resources and workflows and
+will need to use the unmanaged approach instead. This is due to the fact that managed dependent resources always get
+their associated primary resource from the underlying informer event source cache.
+
+#### Additional remarks
+
+As shown in the integration tests, there is no optimistic locking used when updating the
+[resource](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework/src/test/java/io/javaoperatorsdk/operator/baseapi/statuscache/internal/StatusPatchCacheReconciler.java#L41)
+(in other words `metadata.resourceVersion` is set to `null`). This is desired since you don't want the patch to fail on
+update.
+
+In addition, you can configure the [Fabric8 client retry](https://github.com/fabric8io/kubernetes-client?tab=readme-ov-file#configuring-the-client).

operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/PrimaryUpdateAndCacheUtils.java

@@ -0,0 +1,225 @@
+package io.javaoperatorsdk.operator.api.reconciler;
+
+import java.util.function.Supplier;
+import java.util.function.UnaryOperator;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import io.fabric8.kubernetes.api.model.HasMetadata;
+import io.fabric8.kubernetes.client.dsl.base.PatchContext;
+import io.fabric8.kubernetes.client.dsl.base.PatchType;
+import io.javaoperatorsdk.operator.api.reconciler.support.PrimaryResourceCache;
+import io.javaoperatorsdk.operator.processing.event.ResourceID;
+
+/**
+ * Utility methods to patch the primary resource state and store it to the related cache, to make
+ * sure that fresh resource is present for the next reconciliation. The main use case for such
+ * updates is to store state is resource status. Use of optimistic locking is not desired for such
+ * updates, since we don't want to patch fail and lose information that we want to store.
+ */
+public class PrimaryUpdateAndCacheUtils {
+
+  private PrimaryUpdateAndCacheUtils() {}
+
+  private static final Logger log = LoggerFactory.getLogger(PrimaryUpdateAndCacheUtils.class);
+
+  /**
+   * Makes sure that the up-to-date primary resource will be present during the next reconciliation.
+   * Using update (PUT) method.
+   *
+   * @param primary resource
+   * @param context of reconciliation
+   * @return updated resource
+   * @param <P> primary resource type
+   */
+  public static <P extends HasMetadata> P updateAndCacheStatus(P primary, Context<P> context) {
+    logWarnIfResourceVersionPresent(primary);
+    return patchAndCacheStatus(
+        primary, context, () -> context.getClient().resource(primary).updateStatus());
+  }
+
+  /**
+   * Makes sure that the up-to-date primary resource will be present during the next reconciliation.
+   * Using JSON Merge patch.
+   *
+   * @param primary resource
+   * @param context of reconciliation
+   * @return updated resource
+   * @param <P> primary resource type
+   */
+  public static <P extends HasMetadata> P patchAndCacheStatus(P primary, Context<P> context) {
+    logWarnIfResourceVersionPresent(primary);
+    return patchAndCacheStatus(
+        primary, context, () -> context.getClient().resource(primary).patchStatus());
+  }
+
+  /**
+   * Makes sure that the up-to-date primary resource will be present during the next reconciliation.
+   * Using JSON Patch.
+   *
+   * @param primary resource
+   * @param context of reconciliation
+   * @return updated resource
+   * @param <P> primary resource type
+   */
+  public static <P extends HasMetadata> P editAndCacheStatus(
+      P primary, Context<P> context, UnaryOperator<P> operation) {
+    logWarnIfResourceVersionPresent(primary);
+    return patchAndCacheStatus(
+        primary, context, () -> context.getClient().resource(primary).editStatus(operation));
+  }
+
+  /**
+   * Makes sure that the up-to-date primary resource will be present during the next reconciliation.
+   *
+   * @param primary resource
+   * @param context of reconciliation
+   * @param patch free implementation of cache
+   * @return the updated resource.
+   * @param <P> primary resource type
+   */
+  public static <P extends HasMetadata> P patchAndCacheStatus(
+      P primary, Context<P> context, Supplier<P> patch) {
+    var updatedResource = patch.get();
+    context
+        .eventSourceRetriever()
+        .getControllerEventSource()
+        .handleRecentResourceUpdate(ResourceID.fromResource(primary), updatedResource, primary);
+    return updatedResource;
+  }
+
+  /**
+   * Makes sure that the up-to-date primary resource will be present during the next reconciliation.
+   * Using Server Side Apply.
+   *
+   * @param primary resource
+   * @param freshResourceWithStatus - fresh resource with target state
+   * @param context of reconciliation
+   * @return the updated resource.
+   * @param <P> primary resource type
+   */
+  public static <P extends HasMetadata> P ssaPatchAndCacheStatus(
+      P primary, P freshResourceWithStatus, Context<P> context) {
+    logWarnIfResourceVersionPresent(freshResourceWithStatus);
+    var res =
+        context
+            .getClient()
+            .resource(freshResourceWithStatus)
+            .subresource("status")
+            .patch(
+                new PatchContext.Builder()
+                    .withForce(true)
+                    .withFieldManager(context.getControllerConfiguration().fieldManager())
+                    .withPatchType(PatchType.SERVER_SIDE_APPLY)
+                    .build());
+
+    context
+        .eventSourceRetriever()
+        .getControllerEventSource()
+        .handleRecentResourceUpdate(ResourceID.fromResource(primary), res, primary);
+    return res;
+  }
+
+  /**
+   * Patches the resource and adds it to the {@link PrimaryResourceCache}.
+   *
+   * @param primary resource
+   * @param freshResourceWithStatus - fresh resource with target state
+   * @param context of reconciliation
+   * @param cache - resource cache managed by user
+   * @return the updated resource.
+   * @param <P> primary resource type
+   */
+  public static <P extends HasMetadata> P ssaPatchAndCacheStatus(
+      P primary, P freshResourceWithStatus, Context<P> context, PrimaryResourceCache<P> cache) {
+    logWarnIfResourceVersionPresent(freshResourceWithStatus);
+    return patchAndCacheStatus(
+        primary,
+        cache,
+        () ->
+            context
+                .getClient()
+                .resource(freshResourceWithStatus)
+                .subresource("status")
+                .patch(
+                    new PatchContext.Builder()
+                        .withForce(true)
+                        .withFieldManager(context.getControllerConfiguration().fieldManager())
+                        .withPatchType(PatchType.SERVER_SIDE_APPLY)
+                        .build()));
+  }
+
+  /**
+   * Patches the resource with JSON Patch and adds it to the {@link PrimaryResourceCache}.
+   *
+   * @param primary resource
+   * @param context of reconciliation
+   * @param cache - resource cache managed by user
+   * @return the updated resource.
+   * @param <P> primary resource type
+   */
+  public static <P extends HasMetadata> P editAndCacheStatus(
+      P primary, Context<P> context, PrimaryResourceCache<P> cache, UnaryOperator<P> operation) {
+    logWarnIfResourceVersionPresent(primary);
+    return patchAndCacheStatus(
+        primary, cache, () -> context.getClient().resource(primary).editStatus(operation));
+  }
+
+  /**
+   * Patches the resource with JSON Merge patch and adds it to the {@link PrimaryResourceCache}
+   * provided.
+   *
+   * @param primary resource
+   * @param context of reconciliation
+   * @param cache - resource cache managed by user
+   * @return the updated resource.
+   * @param <P> primary resource type
+   */
+  public static <P extends HasMetadata> P patchAndCacheStatus(
+      P primary, Context<P> context, PrimaryResourceCache<P> cache) {
+    logWarnIfResourceVersionPresent(primary);
+    return patchAndCacheStatus(
+        primary, cache, () -> context.getClient().resource(primary).patchStatus());
+  }
+
+  /**
+   * Updates the resource and adds it to the {@link PrimaryResourceCache}.
+   *
+   * @param primary resource
+   * @param context of reconciliation
+   * @param cache - resource cache managed by user
+   * @return the updated resource.
+   * @param <P> primary resource type
+   */
+  public static <P extends HasMetadata> P updateAndCacheStatus(
+      P primary, Context<P> context, PrimaryResourceCache<P> cache) {
+    logWarnIfResourceVersionPresent(primary);
+    return patchAndCacheStatus(
+        primary, cache, () -> context.getClient().resource(primary).updateStatus());
+  }
+
+  /**
+   * Updates the resource using the user provided implementation anc caches the result.
+   *
+   * @param primary resource
+   * @param cache resource cache managed by user
+   * @param patch implementation of resource update*
+   * @return the updated resource.
+   * @param <P> primary resource type
+   */
+  public static <P extends HasMetadata> P patchAndCacheStatus(
+      P primary, PrimaryResourceCache<P> cache, Supplier<P> patch) {
+    var updatedResource = patch.get();
+    cache.cacheResource(primary, updatedResource);
+    return updatedResource;
+  }
+
+  private static <P extends HasMetadata> void logWarnIfResourceVersionPresent(P primary) {
+    if (primary.getMetadata().getResourceVersion() != null) {
+      log.warn(
+          "The metadata.resourceVersion of primary resource is NOT null, "
+              + "using optimistic locking is discouraged for this purpose. ");
+    }
+  }
+}