Scoverage plugin is not usable in multi-project parallel builds with Gradle 6.7+

[netvl]

Please check this Gradle issue: gradle/gradle#15730

In short, if org.gradle.parallel=true project property is set, using the Scoverage plugin in a multi-project build with compilation dependencies between projects will result in an exception:

A problem occurred configuring project ':first'.
> Could not determine the dependencies of task ':second:compileJava'.
   > Current thread does not hold the state lock for project :second

This project reproduces the error.

The cause of the error is apparently this portion of the plugin's logic:

gradle-scoverage/src/main/groovy/org/scoverage/ScoveragePlugin.groovy

Lines 159 to 161 in f92c942

	def originalCompilationDependencies = recursiveDependenciesOf(compileTask).findAll {
	it instanceof ScalaCompile
	}

which causes some internal invariants in Gradle to fail due to cross-project access.

[russell-bie]

We also encountered this issue in our project. When the scoverage plugin is disabled, everything works file.

[DieBauer]

I'm running into the same issue. This keeps me from upgrading gradle to 6.7. Parallel and Scoverage don't play nice anymore :(

[maiflai]

Any thoughts @eyalroth ? I wonder if this means we just have to instrument the main source set, rather than creating a separate configuration.

[eyalroth]

Sorry for taking so long to reply.

There are two reasons we discover task dependencies recursively (which causes this error in conjunction with --parallel):

1. To make running without normal compilation work, we have to make the scoverage-compile tasks depend on one another.

Trying to make them depend only on directly dependent projects instead of looking for 2+ depth dependencies will fail when in-between projects have no scoverage.

We could make it so that this lookup will happen only if running without normal compilation is selected, and then only that mode will be incompatible with parallel execution, which is fine given that both have the same purpose of improving build times.

However, this may require a change to how this feature is enabled, as I faintly remember that accessing the CLI arguments in the configuration stage is not so easy or perhaps discouraged.

2. To make sure that test tasks of a project run after the report tasks of dependent projects, as we don't want that these tests will affect the report of "inner" projects.

This is something that I'm guessing would be hard to give up, unlike the option for running without normal compilation.

I wonder whether there is a way to run the configuration in a single thread even in the parallel mode.

Edit: A bit of a correction regarding 2) -- it seems that this is only relevant to running without normal compilation as well (it's been awhile since I worked with this code).

In the default mode, the classpath of tests consists of the jars of their dependent projects, and these jars contain the output of the normal compilation, and so running the tests will not affect the coverage of their dependent projects. However, without normal compilation, the jars contain the scoverage instrumented classes (since the "normal" classes were never compiled), and so the tests do affect the coverage of their dependent projects.

[netvl]

I wonder if it is possible to follow the suggestion in the Gradle repo ticket (gradle/gradle#15730) and use configurations to share instrumented code. Basically, as far as I understand it, the idea is to rely on variant-aware dependency selection, which looks like this (in pseudocode):

// Producer project

configurations {
    register("scoverageRuntimeElements") {
        canBeResolved = false
        canBeConsumed = true

        extendsFrom(implementation, runtimeOnly)

        attributes {
            "libraryelements" -> "classes"
            "instrumentation" -> "scoverage"
        }

        outgoing.artifact(scoverageClassesDir) {
            builtBy(scoverageCompileTask)
        }
    }
}

// Consumer project

dependencies {
    implementation(project(":producer"))
}

configurations {
    register("scoverageRuntimeClasspath") {
        canBeResolved = true
        canBeConsumed = false

        extendsFrom(implementation, runtimeOnly)

        attributes {
            "instrumentation" -> "scoverage"
        }
    }
}

In this setup, resolving the scoverageRuntimeClasspath inside the consumer project will result in all external dependencies and scoverage-compiled classes in the producer project. This is because attributes defined on a "resolvable" configuration inside the consumer are used essentially as a filter on dependencies. In this case, even though the producer project exposes lots of variants via several of its configurations (the JAR variant, the resources variant, the class directories variant, etc), because we specify that we want a variant with a particular attribute ("instrumentation" -> "scoverage"), then that's what going to end up inside the scoverageRuntimeClasspath configuration. And from what I understand, this kind of logic will propagate through project dependency chains, meaning that all scoverage tasks will "see" only instrumented classes through all chains of project dependencies.

Then, if the scoverage plugin sets up the test tasks to use this configuration as their classpath instead of the "default" one, I believe that it will kind of automatically result in compilation of only instrumented classes if the user only wants to run tests and compute coverage. If the user wants to publish their code, then because all of the publishing mechanism depends on the "regular" variants (specifically those provided by the runtimeElements/apiElements configurations), they will get the "regular" compilation running too.

[eyalroth]

@netvl The question is whether this will solve the problem of needing to discover tasks that are 2+ levels into the chain list, which is the only reason the plugin (in no-normal-compile mode) is incompatible with parallel builds.

The reason this is important is so support cases where projects in the middle of the chain do not have the scoverage plugin applied to them:

:consumer2 -> :consumer1NoScoverage -> :producer 

If the plugin doesn't apply on the project in the middle, then :consumer1NoScoverage:compileScala is not configured to depend on :producer:compileScoverageScala.

Perhaps there is a way to make compileScala/compileJava of non-scoverage projects depend on scoverage tasks of their dependency projects (maybe the java/scala plugins depend on some output that scoverage can direct its tasks to), but that still leaves the problem of test tasks needing to run after report tasks of their dependencies.

For instance, in the example above, :consumer2:test must run after :producer:reportScoverage, so that the tests of consumers will not affect the coverage report of a producer project (some may actually want that behavior in which case we will need to support this additional feature/mode).