rules_scala coverage doesn't work with bazel 0.24.0 and later

[iirina]

bazel-0.24.0-linux-x86_64 coverage --extra_toolchains="//test/coverage:enable_code_coverage_aspect" //test/coverage/...

outputs an empty bazel-testlogs/test/coverage/test-all/coverage.dat

[iirina]

I'm investigating this and opened the issue to keep track of what I find.

cc @lberki @ittaiz @johnynek

[iirina]

In bazel 0.24 the flag --incompatbile_java_coverage was switched and its functionality was removed, so --noincompatible_java_coverage has no effect. Investigating further.

[iirina]

The test log has no errors. The intermediate jcov.dat file is empty.

Running with --instrumentation_filter=//test/coverage/ triggers a failures instead of a successful build:

java.util.zip.ZipException: error in opening zip file
        at java.util.zip.ZipFile.open(Native Method)
        at java.util.zip.ZipFile.<init>(ZipFile.java:230)
        at java.util.zip.ZipFile.<init>(ZipFile.java:155)
        at java.util.jar.JarFile.<init>(JarFile.java:167)
        at java.util.jar.JarFile.<init>(JarFile.java:131)
        at com.google.testing.coverage.JacocoCoverageRunner.addEntriesToExecPathsSet(JacocoCoverageRunner.java:285)
        at com.google.testing.coverage.JacocoCoverageRunner.createPathsSet(JacocoCoverageRunner.java:270)
        at com.google.testing.coverage.JacocoCoverageRunner.createReport(JacocoCoverageRunner.java:149)
        at com.google.testing.coverage.JacocoCoverageRunner.create(JacocoCoverageRunner.java:142)
        at com.google.testing.coverage.JacocoCoverageRunner$2.run(JacocoCoverageRunner.java:518)

looks like an error that should be fixed in more recent bazel versions

[lberki]

I seem to remember trying with 0.28 and finding the same breakage.

[iirina]

What breakage were you seeing? The test itself doesn't use instrumentation filter and it only fails because of mismatched expected and actual coverage report.

[lberki]

test_coverage_on? I thought that the output error indicated completely broken coverage collection in 0.28, but maybe I misread it?

[iirina]

With 0.27 and 0.28 there is also a mismatch between the jacoco version in rules_scala and in bazel, so that must be fixed as well.

test_coverage_on? I thought that the output error indicated completely broken coverage collection in 0.28, but maybe I misread it?

Yes but the actual culprit may be different. Running the test as is doesn't offer any details about the breakages other than "it doesn't work".

[iirina]

Too many changes overflow. 😞 Since 0.24 JacocoCoverageRunner moved out of bazel to remote_java_tools and since 0.25 the CoverageOutputGenerator moved out of bazel to another remote repository, which makes this hard to debug.

[iirina]

I'll try to look more into this when I have more time, since the tests are now passing at head. For now I'm a bit swamped with bazel 1.0 work.

[lberki]

Yeah, let's not do more work than is strictly necessary for Bazel 1.0.

I rebased #793 and I hope that'll be enough to turn Travis green.

[d-haxton]

Hi @iirina I also investigated this and had a PR be merged 4 days ago (at the time of writing) which should have scala only coverage working.

#780

I got blocked to the point where referencing a java_library breaks coverage (emits no errors), but if it's solely scala_libs/ test it should work.

[iirina]

Hi @d-haxton I saw your PR and was very grateful you made it work for Scala! Otherwise we'd be in a serious fire fighting mode right now :)

There's some information lost somewhere, but didn't figure out where.

[systemlogic]

@d-haxton @iirina
we are using bazel 0.24 and want to leverage the scala code coverage for our repository. I know to use functionality either we have to downgrade or wait for fix in 0.29.0. Are you guys are providing fix in latest version.
Mostly I am seeing connection reset by peer with 0.29.0 rc-4

ERROR: An error occurred during the fetch of repository 'remote_java_tools_linux':
   java.io.IOException: Error downloading [https://mirror.bazel.build/bazel_java_tools/releases/javac11/v4.0/java_tools_javac11_linux-v4.0.zip] to /home/harora37/.cache/bazel/_bazel_harora37/4614b08a42facc1b3b190dce93d1ca57/external/remote_java_tools_linux/java_tools_javac11_linux-v4.0.zip: Connection reset
INFO: Call stack for the definition of repository 'remotejdk11_linux' which is a http_archive (rule definition at /home/harora37/.cache/bazel/_bazel_harora37/4614b08a42facc1b3b190dce93d1ca57/external/bazel_tools/tools/build_defs/repo/http.bzl:292:16):
 - /DEFAULT.WORKSPACE.SUFFIX:200:1
ERROR: /home/harora37/.cache/bazel/_bazel_harora37/4614b08a42facc1b3b190dce93d1ca57/external/bazel_tools/tools/jdk/BUILD:351:1: @bazel_tools//tools/jdk:remote_toolchain depends on @remote_java_tools_linux//:toolchain in repository @remote_java_tools_linux which failed to fetch. no such package '@remote_java_tools_linux//': java.io.IOException: Error downloading [https://mirror.bazel.build/bazel_java_tools/releases/javac11/v4.0/java_tools_javac11_linux-v4.0.zip] to /home/harora37/.cache/bazel/_bazel_harora37/4614b08a42facc1b3b190dce93d1ca57/external/remote_java_tools_linux/java_tools_javac11_linux-v4.0.zip: Connection reset
ERROR: Analysis of target '//java/com/ibm/cedp/access:CredentialFile' failed; build aborted: Analysis failed

bazel-bin: /home/harora37/.cache/bazel/_bazel_harora37/4614b08a42facc1b3b190dce93d1ca57/execroot/__main__/bazel-out/k8-fastbuild/bin
bazel-genfiles: /home/harora37/.cache/bazel/_bazel_harora37/4614b08a42facc1b3b190dce93d1ca57/execroot/__main__/bazel-out/k8-fastbuild/bin
bazel-testlogs: /home/harora37/.cache/bazel/_bazel_harora37/4614b08a42facc1b3b190dce93d1ca57/execroot/__main__/bazel-out/k8-fastbuild/testlogs
character-encoding: file.encoding = ISO-8859-1, defaultCharset = ISO-8859-1
command_log: /home/harora37/.cache/bazel/_bazel_harora37/4614b08a42facc1b3b190dce93d1ca57/command.log
committed-heap-size: 666MB
execution_root: /home/harora37/.cache/bazel/_bazel_harora37/4614b08a42facc1b3b190dce93d1ca57/execroot/__main__
gc-count: 8
gc-time: 76ms
install_base: /home/harora37/.cache/bazel/_bazel_harora37/install/aa14b6596814077c8981e309029a019e
java-home: /home/harora37/.cache/bazel/_bazel_harora37/install/aa14b6596814077c8981e309029a019e/_embedded_binaries/embedded_tools/jdk
java-runtime: OpenJDK Runtime Environment (build 11.0.2+7-LTS) by Azul Systems, Inc.
java-vm: OpenJDK 64-Bit Server VM (build 11.0.2+7-LTS, mixed mode) by Azul Systems, Inc.
max-heap-size: 8415MB
output_base: /home/harora37/.cache/bazel/_bazel_harora37/4614b08a42facc1b3b190dce93d1ca57
output_path: /home/harora37/.cache/bazel/_bazel_harora37/4614b08a42facc1b3b190dce93d1ca57/execroot/__main__/bazel-out
package_path: %workspace%
release: release 0.29.0rc2
repository_cache: /home/harora37/.cache/bazel/_bazel_harora37/cache/repos/v1
server_log: /home/harora37/.cache/bazel/_bazel_harora37/4614b08a42facc1b3b190dce93d1ca57/java.log.harora37.harora37.log.java.20190828-095118.7615
server_pid: 7615
used-heap-size: 86MB

[systemlogic]

@iirina when and where the fix will be available. Can i try some RC and let you know ?