Leader Election 'Cannot Receive Event After...'

[coltmcnealy-lh]

Bug Report

Receiving (potentially harmless...?) stacktrace in the follower pod with a 2-replica leader election setup.

What did you see instead? Under which circumstances?

We saw the following stack trace from the follower pod (the one that wasn't the leader). The stack trace repeated every few seconds for 2-3 minutes and then stopped. The leader pod reconciled our custom resources correctly.

19:21:57 ERROR SharedProcessor - v1/namespaces/littlehorse/pods failed invoking InformerEventSource{resourceClass: Pod} event handler: Cannot receive event after a delete event received
java.lang.IllegalStateException: Cannot receive event after a delete event received
	at io.javaoperatorsdk.operator.processing.event.ResourceState.markEventReceived(ResourceState.java:83) ~[operator.jar:?]
	at io.javaoperatorsdk.operator.processing.event.EventProcessor.markEventReceived(EventProcessor.java:191) ~[operator.jar:?]
	at io.javaoperatorsdk.operator.processing.event.EventProcessor.handleEventMarking(EventProcessor.java:180) ~[operator.jar:?]
	at io.javaoperatorsdk.operator.processing.event.EventProcessor.handleEvent(EventProcessor.java:98) ~[operator.jar:?]
	at io.javaoperatorsdk.operator.processing.event.source.informer.InformerEventSource.lambda$propagateEvent$3(InformerEventSource.java:210) ~[operator.jar:?]
	at java.lang.Iterable.forEach(Iterable.java:75) ~[?:?]
	at io.javaoperatorsdk.operator.processing.event.source.informer.InformerEventSource.propagateEvent(InformerEventSource.java:200) ~[operator.jar:?]
	at io.javaoperatorsdk.operator.processing.event.source.informer.InformerEventSource.onAddOrUpdate(InformerEventSource.java:175) ~[operator.jar:?]
	at io.javaoperatorsdk.operator.processing.event.source.informer.InformerEventSource.onUpdate(InformerEventSource.java:136) ~[operator.jar:?]
	at io.javaoperatorsdk.operator.processing.event.source.informer.InformerEventSource.onUpdate(InformerEventSource.java:73) ~[operator.jar:?]
	at io.fabric8.kubernetes.client.informers.impl.cache.ProcessorListener$UpdateNotification.handle(ProcessorListener.java:92) ~[operator.jar:?]
	at io.fabric8.kubernetes.client.informers.impl.cache.ProcessorListener.add(ProcessorListener.java:50) ~[operator.jar:?]
	at io.fabric8.kubernetes.client.informers.impl.cache.SharedProcessor.lambda$distribute$0(SharedProcessor.java:91) ~[operator.jar:?]
	at io.fabric8.kubernetes.client.informers.impl.cache.SharedProcessor.lambda$distribute$1(SharedProcessor.java:114) ~[operator.jar:?]
	at io.fabric8.kubernetes.client.utils.internal.SerialExecutor.lambda$execute$0(SerialExecutor.java:58) ~[operator.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
	at java.lang.Thread.run(Thread.java:833) [?:?]

[csviri]

This is relevant, thank you @coltmcnealy-lh !