Discuss new Framework API structure

[daviwil]

This issue is intended to be a discussion point for the new PowerShell Editor Services API model. The idea here is that Editor Services will now become a framework for adding new capabilities to editors that use it. The overall goal being that most non-core functionality will be enabled through modules that plug into the Editor Services APIs. PSScriptAnalyzer, Plaster, and other modules will add functionality directly through these APIs rather than PSES having to know about them and manage their integration.

Interfaces

All of these interface and method are subject to change as we develop the ideas further.

• IWorkspace - Provides access to the files in the current workspace and events for when files are opened, closed, changed, or saved
• ILogger - Writing logs to the Editor Services log. Important for module integrations to log their activities
• IFileMarkers - Adding markers to files, like those from PSScriptAnalyzer. Also providing quick fixes for these markers
• IMessageHandlers - Registering message handlers for both Language Server Protocol messages and custom message types
• ICodeLensProvider - Providing "code lenses" at arbitrary locations in the script which provide information and launch actions at that point
• IEditorCommands - Registering extension commands in the editor which show up in the command palette
• ICompletionsProvider - Providing custom completions (and snippets) in addition to the built-in completions and snippets
• IFileSymbols - Gathering information about symbols in a particular file
• IWorkspaceSymbols - Gathering information about symbols across the workspace
• IWindowMessages (?) - Showing messages in the editor window (ShowErrorMessage, etc)
• IWindowPrompts - Showing prompts natively in the editor's UI, mirrors the prompting functionality in the PSHostUserInterface API
• IDebuggerCommmands - Provides commands for the debugger, like breaking, stepping, continuing, etc
• IDebuggerEvents - Exposes events for the debugger, primarily around stopping and resuming
• IRunspaceEvents - Exposes events for when the active runspace changes, like when a remote session is established
• ICommandExecution - Commands for executing commands against the current runspace. Also exposes events that give state change information for command execution

TODO: Add more interfaces here

Please feel free to leave comments below to discuss APIs you'd like to see!

[daviwil]

We should also try to comb over the things that we're currently not implementing from the Language Server Protocol and see what coolness we can expose through APIs.

[daviwil]

Also worth mentioning: this new API will be C#-focused but I'd like to expose it to PowerShell scripters in a useful way. Since we leverage async/await a lot in our APIs, it may be slightly challenging to use in PowerShell. Either we should figure out some good patterns for using the async functions from within PowerShell scripts or we should provide some wrapper API classes that make these APIs more scriptable.

Currently the $psEditor object model is a separate set of classes specifically geared toward scripting. We could either continue to maintain that object model or eventually phase it out in favor of exposing the new API interfaces directly to PowerShell. We'll need to do some experimentation to figure out the right approach.

Ideally we'd just have one API interface, but for the aforementioned reasons it may not be the best API for scripters.

[SeeminglyScience]

• IEditorSettings - Provides access to retrieving and changing editor specific settings.

• IContextProvider - Provides information about the current state of the editor, like current file and cursor location.

[rkeithhill]

Do we still need to support PowerShell versions back to v3? That impacts these external PowerShell modules in that they can't use features not available in v3 like classes, enums, .Where{}, ::new(), etc. I know with Plaster I struggled a bit because I develop on Windows 10 where the minimum version of PowerShell is now 5.1. It is really easy to accidentally use a feature not available in v3.

This could be a big value add for PSScriptAnalyzer i.e. if it could gen a diagnostic record when you use a feature or command not available in a min version of PowerShell specified to it (ditto for cross-platform work). And I assume that cross-platform is super-important as well. Again, a bit hard to avoid non-cross platform commands and features. Although I do keep an Ubuntu VM for such testing but no dice on macOS - for me at least. Having cross-platform builds & testing helps. Same for earlier versions of PowerShell. Maybe we should be setting up our module builds to build and test against v3?

Anyway, this is all to say we should be clear on our target versions and platforms and have adequate build & test support.

[daviwil]

I would definitely prefer to keep supporting PSv3 and up, though it's up to the extension module author to decide whether they want to support all of those versions. We're planning to put together some more rules and tooling to make it easy to write backwards-compatible, cross-platform modules but it'll take a bit of time.

Might be worth having a Plaster template to generate a canonical extension module which has some built-in compatibility checks.

[SeeminglyScience]

Would it be worth having a uniform way to disambiguate what an interface is meant to interact with?

For example, at a glance IEditorCommands could mean the command palette in VSCode, the subset of the command palette we call editor commands currently, or PowerShell commands that interact with the editor.

Maybe a separate namespace for interacting with the editor client, PSES itself, and PowerShell?

[daviwil]

Hmmm, that's a really good point. I wonder if a namespace is enough or if we need a consistent naming convention for the interfaces themselves that make it clear which part of the stack they apply to.

The difficulty with the IEditorCommands interface is that some editors will probably allow you to register top-level commands even though VS Code currently doesn't. There may not be a good way to make that clear in the interface's name, and having a separate interface for different editor styles wouldn't be great either. Any thoughts on what we could do there?

I agree though, the naming of the interfaces I've proposed can be improved, I'm definitely interested in any suggestions you have to make them clearer!

[SeeminglyScience]

Also worth mentioning: this new API will be C#-focused but I'd like to expose it to PowerShell scripters in a useful way. Since we leverage async/await a lot in our APIs, it may be slightly challenging to use in PowerShell. Either we should figure out some good patterns for using the async functions from within PowerShell scripts or we should provide some wrapper API classes that make these APIs more scriptable.

Currently the $psEditor object model is a separate set of classes specifically geared toward scripting. We could either continue to maintain that object model or eventually phase it out in favor of exposing the new API interfaces directly to PowerShell. We'll need to do some experimentation to figure out the right approach.

Ideally we'd just have one API interface, but for the aforementioned reasons it may not be the best API for scripters.

Is the intention to make it easier for module developers or easier from the console for all users?

I ask because the problem is easy to solve if you are writing a module. You just have a private function to await and return the result, then all async methods are just an extra command.

On the other hand if the intention is usability from the console or a wider range of PowerShell users, I think the effort would be better spent in making advanced functions/cmdlets that interact with the API. Even if all it does it wait for a method to finish, it's much more likely to be utilized as a command that has help, can be discovered with Get-Command, takes input from the pipeline, etc.

[rkeithhill]

Thinking of Plaster, don't we need some sort of "activation event" that A) tells Plaster that there is an instance of PSES running and B) provides Plaster with an editor object against which to register commands, pop prompts in the UI. In this case, who is responsible for loading the Plaster module into the PSES integrated console? That has to be PSES, right? That could be done via the extension's profile but would we want to always load an extension like Plaster when it isn't used that often? Anyway, it would be interesting to further discuss the whole PSES / non-core module integration & lifecycle.

Regarding async ops, a long time ago I wrote this to make it easier to call WinRT (UWP) APIs from PowerShell. This is a blocking approaching but sometimes that is OK (kind of like node's fooSync methods): https://github.com/rkeithhill/PoshWinRT/blob/master/PoshWinRT/AsyncOperationWrapper.cs

[daviwil]

@SeeminglyScience I definitely agree with the idea of having cmdlet wrappers for the API to make it easier to use in the console or to scripters, that's part of what I'd like to carry over from PSESHelperLibrary :)

People who are familiar with the ISE are pretty comfortable using the $psISE object model as an API so I think the new interfaces should be accessible in a similar model through $psEditor. However, I don't know if it makes sense to try extremely hard to make a single API that caters to both C# and PowerShell, so some level of wrapper will probably be needed.

@rkeithhill PSES will be responsible for loading Plaster, but when Plaster loads it will have to check whether it's being loaded into the PSES host and then register itself with the appropriate interfaces. Basically what you're saying already :) Yep, since Plaster and PSScriptAnalyzer add core functionality to PSES, we'll be loading them by default if they're installed on the machine. PSES will also handle prompting the user to update those modules so that their latest compatible versions will be used.

I really do need to write up something about the more decentralized module design, I'll start doing that as I'm working on this refactoring. The high level points are this:

• PSES will now just be a core framework for modules to plug into and provide functionality to any editor that uses it. By default it will provide functionality it gets directly from PowerShell: core IntelliSense, signature helpers, syntax errors, and host UI integration.
• PSScriptAnalyzer will plug into PSES and provide both script analysis and code formatting behavior through PSES' new "feature provider" APIs. At some point later, all of our symbol analysis code will come from that module or another one we create that provides core language intelligence features to both PSES and PSSA.
• Plaster will plug into PSES and provide both project/item templating and a "project model" through a similar model
• Other modules, like added functionality from Tobias' work, will be able to plug in and register additional providers for things like IntelliSense/snippets, hover information, editor commands, code lenses, and other things that the Language Server Protocol enables us to provide
• PSES will expand its Gallery integration to make it easier for users to find, install, load, and update extension modules. It will be able to show changelogs from updated modules so that the PowerShell extension (and other editors) don't need to update so often when only module updates are necessary

Hope that answers @dfinke's question too! Let me know if any of that is unclear.

[TylerLeonhardt]

Hey @daviwil @SeeminglyScience @rkeithhill - nice discussion :) I'm just coming back here to see if I can get an update on where this left off. Any comments?

[SeeminglyScience]

The next thing to be "componentized" was workspaces iirc. Right now there is no publically accessible way to get a ScriptFile object. This is required for the other component API's (like getting symbols and code lenses) to be accessible from extension modules.

More specifically, API's from the Microsoft.PowerShell.EditorServices.Workspace (not $psEditor.Workspace) class need to be made into a component and added to the component registry in $psEditor.Components.

Also the biggest limitation I feel when I'm working on ESCS is the lack of ability to add refactoring options. I feel like editor commands were kind of a stop gap, and they're great for what they are. But being able to directly add "quick fix" options (the lightbulb) would be a game changer for extensibility.