Skip to content

JaCoCo evaluation report is failed during measuring coverage in run_with_coverage.sh script #762

Open
@amandelpie

Description

@amandelpie

Description

The section of the documentation https://github.com/UnitTestBot/UTBotJava/blob/main/docs/jlearch/setup.md#continue-setup
could not be reproduced correctly

The JaCoCo report is not generated.

To Reproduce

  1. Train the minimal model according provided instruction
  2. Run the last step to measure the quality of the model
  3. Obtain a few errors: a few failed tests due to the row ./gradlew :utbot-junit-contest:test :utbot-junit-contest:jacocoTestReport
  4. If comment first part with running tests, you will obtain the failing of report generation
  5. If run the :utbot-junit-contest:jacocoTestReport with flags --stacktrace you could find the real reason

Expected behavior

Report should be generated

Actual behavior

Got the following stacktrace

Caused by: java.io.IOException: Error while analyzing /home/jupyter-z00663679/UTBotJava/utbot-junit-contest/build/resources/main/lib/utbot-instrumentation-1.0-SNAPSHOT.jar@kotlin/reflect/jvm/internal/impl/serialization/deserialization/builtins/BuiltInsResourceLoader.class.
        at org.jacoco.core.analysis.Analyzer.analyzerError(Analyzer.java:162)
        at org.jacoco.core.analysis.Analyzer.analyzeClass(Analyzer.java:134)
        at org.jacoco.core.analysis.Analyzer.analyzeClass(Analyzer.java:157)
        at org.jacoco.core.analysis.Analyzer.analyzeAll(Analyzer.java:193)
        at org.jacoco.core.analysis.Analyzer.analyzeZip(Analyzer.java:265)
        at org.jacoco.core.analysis.Analyzer.analyzeAll(Analyzer.java:196)
        at org.jacoco.core.analysis.Analyzer.analyzeAll(Analyzer.java:226)
        at org.jacoco.core.analysis.Analyzer.analyzeAll(Analyzer.java:221)
        at org.jacoco.core.analysis.Analyzer.analyzeAll(Analyzer.java:221)
        at org.jacoco.ant.ReportTask.createBundle(ReportTask.java:570)
        at org.jacoco.ant.ReportTask.createReport(ReportTask.java:545)
        at org.jacoco.ant.ReportTask.execute(ReportTask.java:496)
        ... 140 more
Caused by: java.lang.IllegalStateException: Can't add different class with same name: kotlin/reflect/jvm/internal/impl/serialization/deserialization/builtins/BuiltInsResourceLoader
        at org.jacoco.core.analysis.CoverageBuilder.visitCoverage(CoverageBuilder.java:106)
        at org.jacoco.core.analysis.Analyzer$1.visitEnd(Analyzer.java:99)
        at org.objectweb.asm.ClassVisitor.visitEnd(ClassVisitor.java:378)
        at org.jacoco.core.internal.flow.ClassProbesAdapter.visitEnd(ClassProbesAdapter.java:100)
        at org.objectweb.asm.ClassReader.accept(ClassReader.java:722)
        at org.objectweb.asm.ClassReader.accept(ClassReader.java:401)
        at org.jacoco.core.analysis.Analyzer.analyzeClass(Analyzer.java:116)
        at org.jacoco.core.analysis.Analyzer.analyzeClass(Analyzer.java:132)
        ... 150 more


Additional context

The following solution solves the problem:

  1. Change gradle script in the
jacocoTestReport {
    doFirst {
        delete fileTree (dir: "${buildDir}/resources/main/lib")
    }
    reports {
        html.enabled = true
    }
}

It removes the utbot-instrumentation.jar from the analyzing path
2. Run ./gradlew :utbot-junit-contest:assemble
3. Re-run the run_with_coverage.sh script like the following command ./scripts/run_with_coverage.sh antlr 10 "NN_REWARD_GUIDED_SELECTOR models/linear/0" linear0 true
4. The report is found under eval/jacoco

Metadata

Metadata

Assignees

Labels

ctg-bugIssue is a bug

Type

No type

Projects

Status

Todo

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions