Analyze code coverage

[nulltoken]

• Should be triggered as part of the scheduled build upon the merge of a PR
• See https://github.com/csMACnz/Coveralls.net-Samples/tree/xunit-opencover-appveyor for inspiration

[nulltoken]

Relevant:

• http://blog.stephencleary.com/2015/03/continuous-integration-code-coverage-open-source-net-coreclr-projects.html
• https://github.com/csMACnz/BCLExtensions/blob/master/build.tasks.ps1
• https://github.com/csMACnz/coveralls.net
• https://github.com/coveralls-net/coveralls.net

[dahlbyk]

Not sure why coverage depends on $Env:SHOULD_PACKAGE_NUGET_ARTIFACT?

[nulltoken]

You're right. The idea is to analyze on merges. This also happen to be when we generate NuGet packages.

I think the variable is badly named. What would you call it?

[dahlbyk]

Personally I'd add a separate SHOULD_RUN_COVERALLS variable.

[nulltoken]

Done.

[csMACnz]

The problem is that appveyor removes secure token variables on PR builds. The args list becomes
... --repoToken --commitId $env:...
Which is invalid syntax. I'm still looking at how to solve this usage error ao it becomes more obvious.

If you look at my other projects, I wrap it in a condition to check for the PR build environment variable

[nulltoken]

@csMACnz Thanks for your answer.

I've pushed this as a separate branch (see ntk/coverage...coveralls).

The log still displays the same usage pattern.

However, this time it also fails with a Command exited with code 1 message, still without any troubleshooting message. Could it be possible that the Console.Error.WriteLine() calls aren't intercepted by AppVeyor?

Another question. When the Usage is displayed, the possible options aren't. Is that expected?

/cc @FeodorFitsner

[FeodorFitsner]

You mean AppVeyor swallows Console.Error.WriteLine() from .NET console app?

[nulltoken]

@FeodorFitsner It might. By taking a look at source code it looks like at least one of them should be output in this log.

It's fairly possible I may be completely wrong about this though.

[csMACnz]

Your variables don't match so its still the same problem. You declare coveralls_token then use COVERALLS_REPO_TOKEN.
Hopefully making them the same fixes your problem.

Also, the docopts.net library I use is printing the usage statements, and returning the exit code. That is why none of my console.error.writeline calls show up.

[nulltoken]

Your variables don't match so its still the same problem. You declare coveralls_token then use COVERALLS_REPO_TOKEN.

@csMACnz 💎 Thanks a lot for having pointed me at this mistake!

@dahlbyk @jamill The coveralls branch code coverage has been successfully pushed up to coveralls.io. One can peek at the published analysis at https://coveralls.io/r/libgit2/libgit2sharp

I'm going to update the appveyor.yml file with some Console output in order to ease the troubleshooting of the build steps being run. I'll ping you when it's done.

@csMACnz Amazingly neat package and AWESOME support ‼️

[nulltoken]

TODO:

• Maybe not consider files under LibGit2Sharp.Tests namespace for coverage analysis (see Constants.cs)
• Correctly configure the project root path (c:/projects/libgit2sharp/)

[dahlbyk]

Maybe not consider files under LibGit2Sharp.Tests namespace for coverage analysis (see Constants.cs)

I see you've already done this (cb5d495) but 👍 anyway.

Correctly configure the project root path (c:/projects/libgit2sharp/)

This looks correct now as well.

Coverage numbers look correct. Low coverage on exceptions makes it kind of hard to identify actual gaps in coverage. I noticed most of our "for mocking purposes" ctors were untested, so just pushed a commit to coveralls to see how many "false negatives" we can avoid by actually trying to create mock instances.

[nulltoken]

I noticed most of our "for mocking purposes" ctors were untested, so just pushed a commit to coveralls to see how many "false negatives" we can avoid by actually trying to create mock instances.

😍 It's just been published!

[nulltoken]

@dahlbyk As this looks like working, I'm going to rewrite now both this PR and the coveralls branch. I'll ping you when done.

[nulltoken]

@dahlbyk Done. Let's wait for the report to be published to coveralls.io. Once it's available, how about having this one merged?

[dahlbyk]

[nulltoken]

@dahlbyk Thanks for the help and the feedback ❤️

I've dropped all the coverage analysis. One should be automatically run during the next scheduled build (for now, daily at 00:00UTC)