[Investigate] Fix PowerShell event handling

[rjmholt]

From PowerShell/PSReadLine#1679 (comment).

PowerShell events when run in a non-nested shell try to phone home back to the runspace in which they were registered, with the event handler calling in from another thread.

The assumption PowerShell makes is that when an action occurs and we pulse the pipeline, if the pipeline isn't nested, it must be on a different thread.

Of course in our case, the pipeline is on our thread and isn't nested, because we're running it under an application. This seems to be an oversight in the PowerShell hosting model and I think it needs to be addressed in PowerShell; any attempt to run a PowerShell pipeline on a thread not dedicated to it will have this issue.

Unfortunately for us though, this is a behaviour we're stuck with in Windows PowerShell and PS 7.0, 7.1 and 7.2, so we have to work out what mitigation we can provide.

There are a couple of possible solutions:

1. We can construct an artificial internal cmdlet as our "top level" pipeline and put all of our actual execution underneath it. This would be relatively simple and would probably only have a bit of overhead, but would mean our pipelines are always nested, which could possibly have side-effects.
2. We could experiment with using reflection to just lie to PowerShell about our top-level pipeline being nested. This would probably hit errors somewhere, but might be worth a try.
3. The PSLocalEventManager class is an internal extension of a public abstract class and most of its methods are virtual. We might conceivably be able to generate an override to this class and set it internally in the engine to redirect the eventing flow back to us. I'm not sure if this is possible, and if it is it would probably be a fair amount of work. But it would be the most "integrated" approach.
4. Abandon the event loop on the runspace thread and instead use it as a pump for a dedicated runspace thread. This would mean we can no longer run delegates on the runspace thread as we currently do (including at startup, when we implicitly run all our code on the runspace thread so that things like PSReadLine work correctly). We could address this by changing the delegate task implementation to use another internal cmdlet that simply executes the given delegate in its EndProcessing() block. But this would require some reimplementation inside the internal host.

It's also worth noting that, as far as we can tell, the ISE doesn't have any awareness of this scenario either. So it's not clear if this is a quiet bug in the ISE that either hasn't been noticed or hasn't been large enough to come to attention.

For now, I think, event handling in PSES isn't important enough to invalidate the work here and this is something I'm hopeful can be fixed later.

[andyleejordan]

It's also worth noting that, as far as we can tell, the ISE doesn't have any awareness of this scenario either. So it's not clear if this is a quiet bug in the ISE that either hasn't been noticed or hasn't been large enough to come to attention.

For now, I think, event handling in PSES isn't important enough to invalidate the work here and this is something I'm hopeful can be fixed later.

I agree with this. I think we can move forward with the caveat that the integrated terminal does not support PowerShell or .NET events (or at least, not well). Technically the integrated terminal isn't a fully-fledged PowerShell process, it's the integration point between the editor and editor services that just so happens to let the user run code in the runspace hosting the editor's services. I know it doesn't quite look like that to end user, but it is what it is, and presumably the ISE is somewhat similar.

[andyleejordan]

Is there a way we could, like, turn off PowerShell events? Throw an error if one tries to be registered? Or would that involve having to do one of the four proposed workarounds?

[rjmholt]

So the underlying event manager is directly available with $ExecutionContext.Events. We'd need to do (3) to override it.

Normally thought, Register-EngineEvent and Register-ObjectEvent would be what most people would use, and we could conceivably override those cmdlets to do something else with them...

[rjmholt]

/cc @daxian-dbw

[andyleejordan]

This seems like a problem where we could get 80% of it working (most use cases, like those two cmdlets) pretty easily (override them to queue the events on our thing), and it's just the last 20% (making all events including .NET-esque ones) is what would be very hard.

[daxian-dbw]

Quoted from PowerShell/PSReadLine#1679 (comment):

In order for PowerShell eventing to work with the dedicated pipeline thread design in PSES, PowerShell needs to allow disabling the pulse pipeline generation and OnIdle events.

I think we should do this in PowerShell engine. Other events can still be generated, and it's up to the hosting application to spin up pulse pipeline to allow event handling. Also, it's up to the hosting application to define when it is idle, and generate OnIdle event in such case.

[andyleejordan]

I think that'd be a great change, and it would fix this for future versions of PowerShell Core, but sadly it'll remain quite broken for Windows PowerShell 😫

[daxian-dbw]

Also, we (maybe me) need to go back review the ISE code closely to understand how it does things, for example, is it using a dedicated pipeline thread? how does it handle the PS eventing? It doesn't need to deal with PSReadLine which makes it a lot simpler, but still good to know how things work with it.