AzureRM.* modules held in use by PSSA when Cluster 0.0.0.1 is scanned, prevents uninstall

[rebro-msft]

Repro steps

Install-Module -Name Cluster -RequiredVersion 0.0.0.1 -Repository PSGallery
# This depends on AzureRM, which in turn depends on 45 AzureRM.* modules.  
# All get downloaded at install.

Invoke-ScriptAnalyzer -Path [PathToCluster] -Recurse -Severity Error

Invoke-ScriptAnalyzer -Path [PathToCluster] -Recurse -CustomerRulePath [someCustomRulesPath]

Uninstall-Module -Name Cluster -RequiredVersion 0.0.0.1
# Succeeds

Uninstall-Module -Name AzureRM.Websites
# One of the dependencies installed.
# Fails to uninstall, says is currently in use.

Expected Behavior

I should be able to uninstall the AzureRM.* modules. I don't expect PSSA to hold on to them.

We are unable to uninstall any of the AzureRM.* modules because something in PSSA is holding on to them. This appears to only happen for the Cluster 0.0.0.1 module.

[bergmeister]

Can you please post the exact error message? Does that happen even if you close PowerShell and try to uninstall in a new session?

[rebro-msft]

PS C:\windows\system32> Uninstall-Module -Name AzureRM.Websites -RequiredVersion 4.0.0
WARNING: The version '4.0.0' of module 'AzureRM.Websites' is currently in use. Retry the operation after closing the applications.
PackageManagement\Uninstall-Package : Module 'AzureRM.Websites' is in currently in use or you don't have the required permissions.
At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\1.6.0\PSModule.psm1:2459 char:21
+ ...        $null = PackageManagement\Uninstall-Package @PSBoundParameters
+                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (Microsoft.Power...ninstallPackage:UninstallPackage) [Uninstall-Package], Exception
    + FullyQualifiedErrorId : ModuleIsInUse,Uninstall-Package,Microsoft.PowerShell.PackageManagement.Cmdlets.UninstallPackage

It succeeds if I close and open a new session.

[bergmeister]

How is this related to PSSA? Guessing from the error message I'd say that you are not elevated (i.e. you need to launch PowerShell as an Administrator).

[bergmeister]

When trying to repro, I can confirm that AzureRM.Websites cannot be uninstalled, however can see no link to PSSA but rather to the AzureRM or PowerShellGet. Why do you even want to uninstall a dependency? The error that I get is:

PackageManagement\Uninstall-Package : No match was found for the specified search criteria and module names 'AzureRM.Websites'.
At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\1.0.0.1\PSModule.psm1:2194 char:21
+ ...        $null = PackageManagement\Uninstall-Package @PSBoundParameters
+                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Microsoft.Power...ninstallPackage:UninstallPackage) [Uninstall-Package], Exception
    + FullyQualifiedErrorId : NoMatchFound,Microsoft.PowerShell.PackageManagement.Cmdlets.UninstallPackage

The repro steps were (with this custom rule being used) on a Win10 VM:

Install-Module -Name Cluster -RequiredVersion 0.0.0.1 -Repository PSGallery
install-module psscriptanalyzer
ipmo PSScriptAnalyzer
Invoke-ScriptAnalyzer -Path 'C:\Program Files\WindowsPowerShell\Modules\cluster' -Recurse -Severity Error
Invoke-ScriptAnalyzer -Path 'C:\Program Files\WindowsPowerShell\Modules\cluster' -Recurse -Severity Error -CustomRulePath $PathToCustomRule
Uninstall-Module -Name Cluster -RequiredVersion 0.0.0.1
Uninstall-Module -Name AzureRM.Websites
get-module -ListAvailable AzureRM.Websites # check it is really still installed -> yes it is
Uninstall-Module -Name AzureRM.Websites # fails -> try other methods (fail as well)
Uninstall-Module -Name 'AzureRM.Websites'
get-module -ListAvailable AzureRM.Websites | uninstall-module
# dump info
$error[0] | select *
writeErrorStream      : True
PSMessageDetails      :
Exception             : System.Exception: No match was found for the specified search criteria and module names 'AzureRM.Websites'.
TargetObject          : Microsoft.PowerShell.PackageManagement.Cmdlets.UninstallPackage
CategoryInfo          : ObjectNotFound: (Microsoft.Power...ninstallPackage:UninstallPackage) [Uninstall-Package], Exception
FullyQualifiedErrorId : NoMatchFound,Microsoft.PowerShell.PackageManagement.Cmdlets.UninstallPackage
ErrorDetails          :
InvocationInfo        : System.Management.Automation.InvocationInfo
ScriptStackTrace      : at Uninstall-Module<Process>, C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\1.0.0.1\PSModule.psm1: line 2194
                        at <ScriptBlock>, <No file>: line 1
PipelineIterationInfo : {0, 1}
$PSVersionTable
Name                           Value
----                           -----
PSVersion                      5.1.16299.98
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.16299.98
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

[rebro-msft]

I was definitely running this as elevated.
When you tried to repro, did you run both a regular scan and a scan with custom rules? Or just the regular scan?
The link to PSSA is that if I don't run Invoke-ScriptAnalyzer on the Cluster module, I am able to uninstall the AzureRM.* modules just fine. To me, this indicates something in PSSA is holding on to them. And I was under the impression that PSSA should never load anything, that it's strictly static analysis. Is that correct?
It's necessary for our security scan to uninstall the dependencies to get us back to a clean state between item scans.

[bergmeister]

I tried to repro an a clean WinServer 2016 machine and it uninstalls fine in the same PowerShell session. Yes, I used both a normal scan and one with this custom rule (see my comment above). Maybe your specific rule is holding on to it?
The only thing that I noticed is that I had to install Cluster using the -AllowNoClobber option because I had the following error when trying to install it the first time: A command with name 'Get-AzureStorageContainerAcl' is already available on this system. This module 'Azure.Storage' may override the existing commands. If you still want to install this module 'Azure.Storage', use -AllowClobber parameter.
Are you using the latest version of PSSA (1.16.1)? Which OS/PowerShell version are you using? To me it sounds rather like that there are other things in your system that make it already start with a non-clean state. I would try to uninstall in order of dependencies. Should everything fail then just cleaning up the corresponding folders in "$env:ProgramFiles\WindowsPowerShell\Modules" might be a workaround. Also, if uninstalling works in a new shell then you could try to perform the uninstall ation in a new powershell process powershell -command 'Uninstall-Module -Name Cluster'

[rebro-msft]

Yep, I've got PSSA 1.16.1 installed.
I thought you were onto something with it being about our custom rules, so I tried the repro steps without the custom rules run. Same issue unfortunately.
Interesting, it didn't prompt me for -AllowClobber. What sort of things should I be looking for on my system that would mess with PSSA like this?

[bergmeister]

I had a look a the Cluster installation and I can say that it declares a dependency on AzureRM, therefore you should uninstall AzureRM and not just AzureRM.Websites.
Furthermore I looked into the installation folder and Cluster 0.0.0.1 ships with the .git, .vscode folder! Also, it seems that the latest version is already 0.0.0.3 and don't understand why you test an outdated version of something that seems to be pre-release. See here for the announcement 2 days ago that the PSGallery and PowerShellGet support PreRelease versioning now, which you might want to consider. The last thing that you could try is to do a remove-module psscriptanalyzer before the uninstall but that's all of the advice that I can give since I cannot really repro it, sorry. Maybe someone else can help. Disclaimer: I am not a mantainer of PSSA, only an active community member.

[rebro-msft]

I should clarify, I work on the PowerShell Gallery team. I implemented the prerelease versioning feature outlined in that blog post :)
The PowerShell Gallery runs a security scan on all published items, regardless if prerelease or not. We have been seeing the above issue in our security scan for all versions of Cluster, not just 0.0.0.1.
Thanks for your help investigating. I think you're right, it's definitely something specific to just the Cluster module. I'll dig into it further and respond back if I find anything.

[bergmeister]

Haha, what a coincidence. Well in that case thanks for implementing this new feature. :-)

[rebro-msft]

You're welcome, hope you find it useful :-)
Whether the issue is in Cluster or PSSA or our system, I think PSSA should analyze the module in a different appdomain or process. Then if something does get loaded somehow, we won't hit this issue.

[rebro-msft]

A coworker talked with Kapil who used to work on PSSA. His findings:

After talking to Kapil, it seems like it's the "Get-Command" calls in PSSA that are causing all the DLLs to load. We verified by (A) Running only one rule which happens to use Get-Command and (B) Running Get-Command manually and verifying the DLLs get loaded.

[Blackbaud-ShaydeNofziger]

@rebro-msft Sorry to revive an old thread, but we're seeing extreme slowness when analyzing script modules that are heavy on AzureRM cmdlets. Are you saying the issue stems from the Get-Command loading the DLLs?

@bergmeister We noticed a huge improvement in script analysis times (34+ minutes down to less than 3 minutes) for our module when uninstalling AzureRM from the build machine. But we're afraid that will mean we aren't getting good analysis coverage on commands it isn't able to find definitions for. Can you help me understand what we might lose in terms of script analysis quality if we decide to proceed with this workaround?

Happy to open a separate issue if requested.

[bergmeister]

@Blackbaud-ShaydeNofziger Although the tool is mostly static is consumes the local session to get more information about available commands, aliases or other custom definitions in the loaded session, various rules use the command/alias cache (around 5-10). Could you give an example so that I could repro your performance problem? The AzureRM module is known to be not the very best in terms of performance and will always be a bit of a bottleneck but half an hour is definitely too much.
The issue of @rebro-msft is a different one, it is about DLLs being held in use that prevent uninstallation

[Blackbaud-ShaydeNofziger]

This is a very basic example but demonstrates what I mean - example.zip

Below are my findings with our module in question. The times I was speaking of were from our build agent, but these results are from my laptop. Please note I am unable to share the codebase in a public forum.

These tests were performed in brand new sessions without any Azure[RM] modules explicitly loaded

With AzureRM Modules Installed

C:\Repos\blackbaud-iaas-host-azure [master ≡]> Measure-Command { Invoke-ScriptAnalyzer -Path .\Blackbaud.Host.Azure\*.ps1 }

Days              : 0
Hours             : 0
Minutes           : 13
Seconds           : 32
Milliseconds      : 269
Ticks             : 8122691194
TotalDays         : 0.00940126295601852
TotalHours        : 0.225630310944444
TotalMinutes      : 13.5378186566667
TotalSeconds      : 812.2691194
TotalMilliseconds : 812269.1194

Without AzureRM Modules Installed

C:\Repos\blackbaud-iaas-host-azure [master ≡]> Measure-Command { Invoke-ScriptAnalyzer -Path .\Blackbaud.Host.Azure\*.ps1 }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 24
Milliseconds      : 388
Ticks             : 243888910
TotalDays         : 0.000282278831018518
TotalHours        : 0.00677469194444444
TotalMinutes      : 0.406481516666667
TotalSeconds      : 24.388891
TotalMilliseconds : 24388.891

Would you like me to open a separate issue?