source index using SourceLink.exe

[ctaggart]

fixes #876, changes based on #1003

https://ci.appveyor.com/project/ctaggart/libgit2sharp/build/3/job/odbyu9n88cjfgw1i#L123

Is there a tool that you know of that would allow someone to check that the rewritten .pdb is valid/consistent?

I've addressed @nulltoken's concern from #465 with a sourcelink checksums --check command you can verify the pdb file source index works.

cinst sourcelink
SourceLink.exe checksums -p 'lib\net40\LibGit2Sharp.pdb' -nf -u -c

[nulltoken]

@ctaggart Hmmm. Something is still wrong with the build process. My bad.

@GeertvanHorrik was nice enough to help us ease the source indexing integration, but I've just discovered that we're not there yet.

Let me explain our build process a bit better.

Tests are run with a specific LEAKS_IDENTIFYING define to make the tests fail would we catch a handle leak.

However, we wouldn't want to publish the NuGet package with the result of this build. Thus we have to rebuild it without the extra define.

Published NuGet package should include the LibGit2sharp commit sha so that we're able to output it when GlobalSettings.Version is invoked. The current build log shows (in purple) the result of this invokation

LibGit2Sharp version = 0.22.0-pre20150322083458-unknown-9bbc8f3 (amd64 - Threads, Https) 

Would everything works as intended, instead of unknown we should see 09e94080 in the line above.

Would you be so kind as to apply the following patch to this PR and see if that solves the issue?

diff --git a/appveyor.yml b/appveyor.yml
index 0fc2107..0201617 100644
--- a/appveyor.yml
+++ b/appveyor.yml
@@ -54,9 +54,6 @@ cache:
 before_build:
 - nuget restore "%APPVEYOR_BUILD_FOLDER%\LibGit2Sharp.sln"

-after_build:
-- ps: sourcelink index -u 'https://raw.githubusercontent.com/libgit2/libgit2sharp/{0}/%var2%' -pr LibGit2Sharp\LibGit2Sharp.csproj -pp Configuration Release -nf LibGit2Sharp\Core\UniqueIdentifier.cs -nf LibGit2Sharp\Properties\AssemblyInfo.cs
-
 build_script:
 - msbuild "%APPVEYOR_BUILD_FOLDER%\LibGit2Sharp.sln" /verbosity:normal /p:Configuration=Release /logger:"C:\Program Files\AppVeyor\BuildAgent\Appveyor.MSBuildLogger.dll" /property:ExtraDefine="LEAKS_IDENTIFYING"

@@ -66,7 +63,7 @@ test_script:

 on_success:
 - ps: |
-    & "$env:APPVEYOR_BUILD_FOLDER\nuget.package\BuildNugetPackage.ps1" "$env:APPVEYOR_REPO_COMMIT"
+    & "$env:APPVEYOR_BUILD_FOLDER\nuget.package\BuildNugetPackage.ps1" -commitSha "$env:APPVEYOR_REPO_COMMIT" -postBuild  { sourcelink index -u 'https://raw.githubusercontent.com/libgit2/libgit2sharp/{0}/%var2%' -pr LibGit2Sharp\LibGit2Sharp.csproj -pp Configuration Release -nf LibGit2Sharp\Core\UniqueIdentifier.cs -nf LibGit2Sharp\Properties\AssemblyInfo.cs }
     Add-Type -Path "$env:APPVEYOR_BUILD_FOLDER\LibGit2Sharp\bin\Release\LibGit2Sharp.dll"
     Write-Host "LibGit2Sharp version = $([LibGit2Sharp.GlobalSettings]::Version)" -ForegroundColor "Magenta"
     #If ($Env:SHOULD_PUBLISH_NUGET_ARTIFACT -eq $True)
diff --git a/nuget.package/BuildNugetPackage.ps1 b/nuget.package/BuildNugetPackage.ps1
index 8eac5e5..6770c14 100644
--- a/nuget.package/BuildNugetPackage.ps1
+++ b/nuget.package/BuildNugetPackage.ps1
@@ -8,7 +8,8 @@

 Param(
     [Parameter(Mandatory=$true)]
-    [string]$commitSha
+    [string]$commitSha,
+    [scriptblock]$postBuild
 )

 $ErrorActionPreference = "Stop"
@@ -51,13 +52,25 @@ function Clean-OutputFolder($folder) {
     }
 }

+# From http://www.dougfinke.com/blog/index.php/2010/12/01/note-to-self-how-to-programmatically-get-the-msbuild-path-in-powershell/
+
+Function Get-MSBuild {
+    $lib = [System.Runtime.InteropServices.RuntimeEnvironment]
+    $rtd = $lib::GetRuntimeDirectory()
+    Join-Path $rtd msbuild.exe
+}
+
 #################

 $root = Split-Path -Parent -Path $MyInvocation.MyCommand.Definition
 $projectPath = Join-Path $root "..\LibGit2Sharp"
+$slnPath = Join-Path $projectPath "..\LibGit2Sharp.sln"

 Remove-Item (Join-Path $projectPath "*.nupkg")

+Clean-OutputFolder (Join-Path $projectPath "bin\")
+Clean-OutputFolder (Join-Path $projectPath "obj\")
+
 # The nuspec file needs to be next to the csproj, so copy it there during the pack operation
 Copy-Item (Join-Path $root "LibGit2Sharp.nuspec") $projectPath

@@ -65,7 +78,14 @@ Push-Location $projectPath

 try {
   Set-Content -Encoding ASCII $(Join-Path $projectPath "libgit2sharp_hash.txt") $commitSha
-  Run-Command { & "$(Join-Path $projectPath "..\Lib\NuGet\Nuget.exe")" Restore "$(Join-Path $projectPath "..\LibGit2Sharp.sln")" }
+  Run-Command { & "$(Join-Path $projectPath "..\Lib\NuGet\Nuget.exe")" Restore "$slnPath" }
+  Run-Command { & (Get-MSBuild) "$slnPath" "/verbosity:minimal" "/p:Configuration=Release" }
+
+  If ($postBuild) {
+    Write-Host -ForegroundColor "Green" "Run post build script..."
+    Run-Command { & ($postBuild) }
+  }
+
   Run-Command { & "$(Join-Path $projectPath "..\Lib\NuGet\Nuget.exe")" Pack -Prop Configuration=Release }
 }
 finally {

Provided I didn't mess up too bad, this should solve the problem

Disclaime: I'm NOT a PowerShell guy, so there may be better ways to achieve this.

[ctaggart]

There was a problem applying that diff:

[nulltoken]

@ctaggart Could you please apply the changes manually?

[ctaggart]

Will do.

[nulltoken]

@ctaggart ❤️ I've locally tested the changes by passing a dummy postbuild scriptblock. However:

• As previously written, I'm not a PowerShell wizard
• I haven't tested the scriptblock with the real sourcelink.rewriting process

[nulltoken]

@ctaggart Surprisngly the postbuild trickery seems to be working. However, the paths may require some tweaking (sorry about that)

Run post build script... 
SourceLink 0.5.0
SourceLink failed with:
  unable to load proj file `LibGit2Sharp\LibGit2Sharp.csproj` with properties: [("Configuration", "Release")], error The project file could not be loaded. Could not find a part of the path 'C:\projects\libgit2sharp\LibGit2Sharp\LibGit2Sharp\LibGit2Sharp.csproj'.  C:\projects\libgit2sharp\LibGit2Sharp\LibGit2Sharp\LibGit2Sharp.csproj
` & ($postBuild) ` failed

[nulltoken]

Maybe passing absolute paths starting with $env:APPVEYOR_BUILD_FOLDER would simplify this?

[ctaggart]

Based on the output, we can see it is running from the project directly. This is because of Push-Location in BuildNuGetPackage.ps1. It is pretty straight forward to run SourceLink.exe from that directory.

Do you have chocolatey installed? If not, install it by running PowerShell as Administrator with:

iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1'))

Then you can install SourceLink with:

cinst SourceLink -version 0.5.0

In a new regular PowerShell window, you can then do

cd C:\Projects\libgit2sharp\LibGit2Sharp
SourceLink.exe index `
  -pr ./LibGit2Sharp.csproj `
  -pp Configuration Release `
  -nf Core\UniqueIdentifier.cs `
  -nf Properties\AssemblyInfo.cs `
  -r .. `
  -u 'https://raw.githubusercontent.com/libgit2/libgit2sharp/{0}/%var2%'

It is easiest to explain what I did by looking at the help for the index command.

The csproj is used to get a list of source files used. We exclude a couple of files that are generated at build time. We say that the repository root is in the parent folder. We say what URL to source index with. SourceLink then verifies that the checksums of the files in the working directory used to create the build match those in the pdb and the git repository. That way, when Visual Studio or other debuggers download the source file from GitHub, the checksum in the pdb and of the download file match.

I'l update the pull request.

[nulltoken]

@ctaggart That worked perfectly ✨ Thanks for the detailed explanation as well.

Could you please uncomment the part that detects if NuGet packages should be published, in appveyor.yaml?

While you're at it, please squash all the commits into one and we'll get this merged!

[ctaggart]

What is the best way to squash? It is not something I've done. I looked up stackoverflow and this is what I came up with:

git reset --soft HEAD~7
git commit -m 'fix libgit2/libgit2sharp#876 adds source indexing using SourceLink.exe'
git push ctaggart sourcelinkexe

Git is hanging on the last statement. Any ideas?

[nulltoken]

Provided your commit has been created, you'd have to force push to update the remote head.

[ctaggart]

I upgraded git and tried again and it looks like it worked.

git version 1.8.5.2.msysgit.0
git version 1.9.5.msysgit.1

[ctaggart]

Both files built with AppVeyor successfully pass the new SourceLink checksums check that downs all the files and compares each md5.

SourceLink.exe checksums -p C:\tmp\LibGit2Sharp.0.22.0-pre20150322083807\lib\net40\LibGit2Sharp.pdb -c -v

[nulltoken]

Published as NuGet pre-release package LibGit2Sharp.0.22.0-pre20150323064819