Skip to content

Pester test Context failures not showing up in Problems window #2438

Closed
@tillig

Description

@tillig

Issue Description

If you have a Pester test where a BeforeAll or AfterAll block fails that failure doesn't show up in the Problems window. Test failures do properly display.

Here's a repro test you can use to try it out:

Describe "Reproduce Issue" {
    Context "Failing BeforeAll in Context block" {
        BeforeAll {
            # In a "real" scenario, this is
            # $pathToNuGet = Get-Command nuget.exe | Select-Object -ExpandProperty Source
            # &mono $pathToNuGet do-some-things-with-nuget.exe
            # to execute NuGet on a Linux/OS X box. The failure I'm seeing in the context
            # happens when mono isn't available, and possibly nuget.exe is also not there.
            $pathToCommand = Get-Command nosuchcommand.exe | Select-Object -ExpandProperty Source
            &othermissingcommand $pathToCommand
        }
    }
    It "has two failures - context and test" {
        $False | Should -Be $True
    }
}

The output of this in the console looks like:

> Executing task: Invoke-Pester -PesterOption @{IncludeVSCodeMarker=$true} <

    ____            __
   / __ \___  _____/ /____  _____
  / /_/ / _ \/ ___/ __/ _ \/ ___/
 / ____/  __(__  ) /_/  __/ /
/_/    \___/____/\__/\___/_/
Pester v4.9.0
Executing all tests in '.'

Executing script /Users/tillig/dev/repro/Repro.Tests.ps1

  Describing Reproduce Issue

    Context Failing BeforeAll in Context block
Get-Command: /Users/tillig/dev/repro/Repro.Tests.ps1:9:30
Line |
   9 |              $pathToCommand = Get-Command nosuchcommand.exe | Select-O …
     |                               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | The term 'nosuchcommand.exe' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the
     | spelling of the name, or if a path was included, verify that the path is correct and try again.

      [-] Error occurred in Context block 0ms
        At /Users/tillig/dev/repro/Repro.Tests.ps1:10 char:14
        The term 'othermissingcommand' is not recognized as the name of a cmdlet, function, script file, or operable program.
        Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
    [-] has two failures - context and test 77ms
      at <ScriptBlock>, /Users/tillig/dev/repro/Repro.Tests.ps1: line 14
      14:         $False | Should -Be $True
      Expected $true, but got $false.
Tests completed in 508ms
Tests Passed: 0, Failed: 2, Skipped: 0, Pending: 0, Inconclusive: 0 

Terminal will be reused by tasks, press any key to close it.

The Problems window does display the test failure - 'has two failures - context and test' but it does NOT display the failure in the test context itself.

If you switch the test above so it's $True | Should -Be $True (so there's an assertion and it passes), only the Context will fail. Re-run the test and you'll see there's nothing in the Problem window.

Attached Logs

I didn't notice anything in the logs that indicated a failure around this, it appears it's a problem matcher issue. I can gather logs if you think it'd help.

Environment Information

Visual Studio Code

Name Version
Operating System Darwin x64 19.2.0
VSCode 1.41.1
PowerShell Extension Version 2020.1.0

PowerShell Information

Name Value
PSVersion 7.0.0-rc.2
PSEdition Core
GitCommitId 7.0.0-rc.2
OS Darwin 19.2.0 Darwin Kernel Version 19.2.0: Sat Nov 9 03:47:04 PST 2019, root:xnu-6153.61.1~20/RELEASE_X86_64
Platform Unix
PSCompatibleVersions 1.0 2.0 3.0 4.0 5.0 5.1.10032.0 6.0.0 6.1.0 6.2.0 7.0.0-rc.2
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
WSManStackVersion 3.0

Visual Studio Code Extensions

Visual Studio Code Extensions(Click to Expand)
Extension Author Version
auto-close-tag formulahendry 0.5.6
auto-rename-tag formulahendry 0.1.1
azure-account ms-vscode 0.8.8
azurerm-vscode-tools msazurermtools 0.8.3
beautify HookyQR 1.5.0
code-runner formulahendry 0.9.15
code-settings-sync Shan 3.4.3
color-info bierner 0.5.1
cpptools ms-vscode 0.26.3
csharp ms-vscode 1.21.9
debugger-for-chrome msjsdiag 4.12.6
dotenv mikestead 1.0.1
dotnet-test-explorer formulahendry 0.7.1
ecdc mitchdenny 1.3.0
EditorConfig EditorConfig 0.14.4
gitlens eamodio 10.2.0
Go ms-vscode 0.12.0
guides spywhere 0.9.3
html-css-class-completion Zignd 1.19.0
java redhat 0.55.1
markdown-preview-enhanced shd101wyy 0.5.1
material-theme zhuangtongfa 3.2.1
msbuild-project-tools tintoy 0.3.3
mssql ms-mssql 1.8.0
nc-gcode ML 0.13.0
ng-template Angular 0.900.6
npm-intellisense christian-kohler 1.3.0
output-colorizer IBM 0.1.2
path-intellisense christian-kohler 1.4.2
polacode pnp 0.3.4
powershell ms-vscode 2020.1.0
python ms-python 2020.1.58038
remote-wsl ms-vscode-remote 0.41.7
restructuredtext lextudio 117.0.0
rewrap stkb 1.9.1
sort-lines Tyriar 1.9.0
sublime-keybindings ms-vscode 4.0.6
terraform mauve 1.4.0
todo-tree Gruntfuggly 0.0.166
vscode-aks-tools ms-kubernetes-tools 0.0.4
vscode-arduino vsciot-vscode 0.2.29
vscode-docker ms-azuretools 0.9.0
vscode-eslint dbaeumer 2.0.15
vscode-icons vscode-icons-team 9.7.0
vscode-java-debug vscjava 0.24.0
vscode-java-dependency vscjava 0.8.0
vscode-java-pack vscjava 0.8.1
vscode-java-test vscjava 0.22.1
vscode-json-stable-stringify TravisIllig 1.1.0
vscode-kubernetes-tools ms-kubernetes-tools 1.0.9
vscode-markdownlint DavidAnson 0.33.0
vscode-maven vscjava 0.20.2
vscode-npm-script eg2 0.3.10
vscode-typescript-tslint-plugin ms-vscode 1.2.3
vscode-yaml redhat 0.6.1
vscodeintellicode VisualStudioExptTeam 1.2.4
xml DotJoshJohnson 2.5.0

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions