var with JDK 13 - CLI throws "NoSuchFieldException: modifiers"

[alisevych]

Description

CLI with JDK 13
Class using var field definition
CLI throws "NoSuchFieldException: modifiers"

To Reproduce

Steps to reproduce the behavior:

1. Define JDK-13 as primary (Path, JAVA_HOME)
2. Download one of the latest CLI from master
3. For simplicity in the folder where utbot-cli*.jar is located :
   Create ExampleString.java using var (feature introduced in Java 9)

public class ExampleString {

    public void stringNewMethod() {
        var text = "Hello!\n This is String defined with var";

        text = text.toLowerCase();
        System.out.println(text);

        text = text.toUpperCase();
        System.out.println(text);
    }
}

4. Compile your files with JDK-13

javac -version

javac ExampleString.class

5. Run utbot-cli to generate tests for it, like that:

java -jar utbot-cli-2022.7.jar generate --source ExampleString.java --classpath "D:\Current\Java" -o ExampleStringTest.java ExampleString

Expected behavior

Tests are supposed to be generated.

Actual behavior

There is the following output in console:

var used, compiled and executed with JDK-13


WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
16:05:00.257 | INFO  | AllInMemoryClassProvider | Loaded: 2 path entries, 46 classes, 194946 bytes
16:05:01.443 | INFO  | UtBotTestCaseGenerator | |> Resuming method ExampleString.stringNewMethod()
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.esotericsoftware.kryo.unsafe.UnsafeUtil (file:/D:/Current/Java/utbot-cli-2022.7.jar) to constructor java.nio.DirectByteBuffer(long,int)
WARNING: Please consider reporting this to the maintainers of com.esotericsoftware.kryo.unsafe.UnsafeUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
16:05:05.078 | ERROR | UtBotSymbolicEngine | Test generation failed on stmt return, symbolic stack trace:
<ExampleString: void stringNewMethod()>
org.utbot.instrumentation.util.ChildProcessError: Error in the child process |> java.lang.ExceptionInInitializerError
        at org.utbot.framework.util.EngineUtilsKt.getAnyInstance(EngineUtils.kt:35)
        at org.utbot.framework.concrete.MockValueConstructor.constructObject(MockValueConstructor.kt:161)
        at org.utbot.framework.concrete.MockValueConstructor.construct(MockValueConstructor.kt:127)
        at org.utbot.framework.concrete.MockValueConstructor.constructStatics(MockValueConstructor.kt:110)
        at org.utbot.framework.concrete.UtExecutionInstrumentation.invoke(UtExecutionInstrumentation.kt:146)
        at org.utbot.framework.concrete.UtExecutionInstrumentation.invoke(UtExecutionInstrumentation.kt:107)
        at org.utbot.instrumentation.process.ChildProcessKt.loop(ChildProcess.kt:133)
        at org.utbot.instrumentation.process.ChildProcessKt.main(ChildProcess.kt:77)
        at org.utbot.instrumentation.process.ChildProcessKt.main(ChildProcess.kt)
Caused by: java.lang.NoSuchFieldException: modifiers
        at java.base/java.lang.Class.getDeclaredField(Class.java:2412)
        at org.utbot.common.Reflection.<clinit>(ReflectionUtil.kt:18)
        ... 9 more

        at org.utbot.instrumentation.ConcreteExecutor.executeAsync(ConcreteExecutor.kt:415) ~[utbot-cli-2022.7.jar:?]
        at org.utbot.instrumentation.ConcreteExecutor$executeAsync$1.invokeSuspend(ConcreteExecutor.kt) ~[utbot-cli-2022.7.jar:?]
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) [utbot-cli-2022.7.jar:?]
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106) [utbot-cli-2022.7.jar:?]
        at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:274) [utbot-cli-2022.7.jar:?]
        at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:84) [utbot-cli-2022.7.jar:?]
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59) [utbot-cli-2022.7.jar:?]
        at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source) [utbot-cli-2022.7.jar:?]
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38) [utbot-cli-2022.7.jar:?]
        at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source) [utbot-cli-2022.7.jar:?]
        at org.utbot.common.ConcurrencyKt.runBlockingWithCancellationPredicate(Concurrency.kt:38) [utbot-cli-2022.7.jar:?]
        at org.utbot.framework.plugin.api.UtBotTestCaseGenerator$generateForSeveralMethods$4.invoke(UtBotTestCaseGenerator.kt:279) [utbot-cli-2022.7.jar:?]
        at org.utbot.framework.plugin.api.UtBotTestCaseGenerator$generateForSeveralMethods$4.invoke(UtBotTestCaseGenerator.kt:56) [utbot-cli-2022.7.jar:?]
        at org.utbot.common.ConcurrencyKt.runIgnoringCancellationException(Concurrency.kt:47) [utbot-cli-2022.7.jar:?]
        at org.utbot.framework.plugin.api.UtBotTestCaseGenerator.generateForSeveralMethods(UtBotTestCaseGenerator.kt:278) [utbot-cli-2022.7.jar:?]
        at org.utbot.framework.plugin.api.UtBotTestCaseGenerator.generateForSeveralMethods$default(UtBotTestCaseGenerator.kt:265) [utbot-cli-2022.7.jar:?]
        at org.utbot.cli.GenerateTestsAbstractCommand.generateTestCases(GenerateTestsAbstractCommand.kt:163) [utbot-cli-2022.7.jar:?]
        at org.utbot.cli.GenerateTestsCommand.run(GenerateTestsCommand.kt:102) [utbot-cli-2022.7.jar:?]
        at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:204) [utbot-cli-2022.7.jar:?]
        at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:213) [utbot-cli-2022.7.jar:?]
        at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:17) [utbot-cli-2022.7.jar:?]
        at com.github.ajalt.clikt.core.CliktCommand.parse(CliktCommand.kt:396) [utbot-cli-2022.7.jar:?]
        at com.github.ajalt.clikt.core.CliktCommand.parse$default(CliktCommand.kt:393) [utbot-cli-2022.7.jar:?]
        at com.github.ajalt.clikt.core.CliktCommand.main(CliktCommand.kt:411) [utbot-cli-2022.7.jar:?]
        at com.github.ajalt.clikt.core.CliktCommand.main(CliktCommand.kt:436) [utbot-cli-2022.7.jar:?]
        at org.utbot.cli.ApplicationKt.main(Application.kt:31) [utbot-cli-2022.7.jar:?]
Caused by: java.lang.ExceptionInInitializerError
        at org.utbot.framework.util.EngineUtilsKt.getAnyInstance(EngineUtils.kt:35) ~[utbot-cli-2022.7.jar:?]
        at org.utbot.framework.concrete.MockValueConstructor.constructObject(MockValueConstructor.kt:161) ~[utbot-cli-2022.7.jar:?]
        at org.utbot.framework.concrete.MockValueConstructor.construct(MockValueConstructor.kt:127) ~[utbot-cli-2022.7.jar:?]
        at org.utbot.framework.concrete.MockValueConstructor.constructStatics(MockValueConstructor.kt:110) ~[utbot-cli-2022.7.jar:?]
        at org.utbot.framework.concrete.UtExecutionInstrumentation.invoke(UtExecutionInstrumentation.kt:146) ~[utbot-cli-2022.7.jar:?]
        at org.utbot.framework.concrete.UtExecutionInstrumentation.invoke(UtExecutionInstrumentation.kt:107) ~[utbot-cli-2022.7.jar:?]
        at org.utbot.instrumentation.process.ChildProcessKt.loop(ChildProcess.kt:133) ~[utbot-cli-2022.7.jar:?]
        at org.utbot.instrumentation.process.ChildProcessKt.main(ChildProcess.kt:77) ~[utbot-cli-2022.7.jar:?]
        at org.utbot.instrumentation.process.ChildProcessKt.main(ChildProcess.kt) ~[utbot-cli-2022.7.jar:?]
Caused by: java.lang.NoSuchFieldException: modifiers
        at java.lang.Class.getDeclaredField(Class.java:2412) ~[?:?]
        at org.utbot.common.Reflection.<clinit>(ReflectionUtil.kt:18) ~[utbot-cli-2022.7.jar:?]
        at org.utbot.framework.util.EngineUtilsKt.getAnyInstance(EngineUtils.kt:35) ~[utbot-cli-2022.7.jar:?]
        at org.utbot.framework.concrete.MockValueConstructor.constructObject(MockValueConstructor.kt:161) ~[utbot-cli-2022.7.jar:?]
        at org.utbot.framework.concrete.MockValueConstructor.construct(MockValueConstructor.kt:127) ~[utbot-cli-2022.7.jar:?]
        at org.utbot.framework.concrete.MockValueConstructor.constructStatics(MockValueConstructor.kt:110) ~[utbot-cli-2022.7.jar:?]
        at org.utbot.framework.concrete.UtExecutionInstrumentation.invoke(UtExecutionInstrumentation.kt:146) ~[utbot-cli-2022.7.jar:?]
        at org.utbot.framework.concrete.UtExecutionInstrumentation.invoke(UtExecutionInstrumentation.kt:107) ~[utbot-cli-2022.7.jar:?]
        at org.utbot.instrumentation.process.ChildProcessKt.loop(ChildProcess.kt:133) ~[utbot-cli-2022.7.jar:?]
        at org.utbot.instrumentation.process.ChildProcessKt.main(ChildProcess.kt:77) ~[utbot-cli-2022.7.jar:?]
        at org.utbot.instrumentation.process.ChildProcessKt.main(ChildProcess.kt) ~[utbot-cli-2022.7.jar:?]
16:05:05.147 | INFO  | Summarization | Diversity execution path percentage: 100
16:05:05.147 | INFO  | Summarization | No execution traces found in test case for method ExampleString,     public void stringNewMethod()
    {
        java.io.PrintStream $r0, $r1;
        ExampleString r2;
        java.lang.String r3, r4, r5;

        r2 := @this: ExampleString;

        r3 = "Hello Baeldung!\nThis is Java 12 article.";

        r4 = virtualinvoke r3.<java.lang.String: java.lang.String toLowerCase()>();

        $r0 = <java.lang.System: java.io.PrintStream out>;

        virtualinvoke $r0.<java.io.PrintStream: void println(java.lang.String)>(r4);

        r5 = virtualinvoke r4.<java.lang.String: java.lang.String toUpperCase()>();

        $r1 = <java.lang.System: java.io.PrintStream out>;

        virtualinvoke $r1.<java.io.PrintStream: void println(java.lang.String)>(r5);

        return;
    }

[sergeypospelov]

Some investigation on #374:

It seems to me, that this problem isn't related to var directly. Such errors can happen in any code if we run it with JDK 13, but on JDK 9-11, it should be OK.

In all places (e.g., in the child process) where we construct concrete (Any?) values from models, we highly depend on withAccessability function allowing us to call private methods with reflection.

This is the code snippet we use:

package org.utbot.common

import org.utbot.common.Reflection.setModifiers
import java.lang.reflect.AccessibleObject
import java.lang.reflect.Field
import java.lang.reflect.Modifier
import sun.misc.Unsafe

object Reflection {
    val unsafe: Unsafe

    init {
        val f: Field = Unsafe::class.java.getDeclaredField("theUnsafe")
        f.isAccessible = true
        unsafe = f.get(null) as Unsafe
    }

    private val modifiersField: Field = Field::class.java.getDeclaredField("modifiers") // problem is here

    init {
        modifiersField.isAccessible = true
    }

    fun setModifiers(field: Field, modifiers: Int) {
        modifiersField.set(field, modifiers)
    }
}

inline fun <R> AccessibleObject.withAccessibility(block: () -> R): R {
    val prevAccessibility = isAccessible
    try {
        isAccessible = true
        return block()
    } finally {
        isAccessible = prevAccessibility
    }
}

The problem is that from Java 12, there is no such field modifiers available via reflection, so the code fails.
The more details can be found on StackOverflow: get-declared-fields-of-java-lang-reflect-fields-in-jdk12

The quick workaround:

Method getDeclaredFields0 = Class.class.getDeclaredMethod("getDeclaredFields0", boolean.class);
getDeclaredFields0.setAccessible(true);
Field[] fields = (Field[]) getDeclaredFields0.invoke(Field.class, false);
Field modifiers = null;
for (Field each : fields) {
    if ("modifiers".equals(each.getName())) {
        modifiers = each;
        break;
    }
}
assertNotNull(modifiers);

and add this to running options:
--add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.lang.reflect=ALL-UNNAMED

Will try this out