Runtime staging compiler crash when creating an object using "new" (scala 3.3.1)

[christophkoch]

Compiler version

3.3.1

Minimized code

import scala.quoted.*
given staging.Compiler = staging.Compiler.make(getClass.getClassLoader)

class A(i: Int)

def f(i: Expr[Int])(using Quotes): Expr[A] = { '{ new A($i) } }

val g: Int => A = staging.run { '{ (i: Int) => ${ f('{i}) } } }

Output

scala> g
dotty.tools.dotc.reporting.UnhandledError: undefined: new A # -1: TermRef(TypeRef(ThisType(TypeRef(ThisType(TypeRef(NoPrefix,module class )),module class )),A),) at quotedFrontend
at dotty.tools.dotc.reporting.ThrowingReporter.doReport(ThrowingReporter.scala:14)
at dotty.tools.dotc.reporting.Reporter.issueUnconfigured(Reporter.scala:158)
at dotty.tools.dotc.reporting.Reporter.go$1(Reporter.scala:181)
at dotty.tools.dotc.reporting.Reporter.issueIfNotSuppressed(Reporter.scala:200)
at dotty.tools.dotc.reporting.Reporter.report(Reporter.scala:203)
at dotty.tools.dotc.report$.error(report.scala:68)
at dotty.tools.dotc.typer.ErrorReporting$.errorType(ErrorReporting.scala:31)
at dotty.tools.dotc.typer.TypeAssigner.assignType(TypeAssigner.scala:298)
at dotty.tools.dotc.typer.TypeAssigner.assignType$(TypeAssigner.scala:16)
at dotty.tools.dotc.typer.Typer.assignType(Typer.scala:116)
at dotty.tools.dotc.ast.tpd$.Apply(tpd.scala:49)
at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readLengthTree$1(TreeUnpickler.scala:1304)
at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readTree(TreeUnpickler.scala:1468)
at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readLengthTree$1(TreeUnpickler.scala:1333)
at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readTree(TreeUnpickler.scala:1468)
at dotty.tools.dotc.core.tasty.TreeUnpickler.unpickle(TreeUnpickler.scala:117)
at dotty.tools.dotc.core.tasty.DottyUnpickler.computeRootTrees(DottyUnpickler.scala:62)
at dotty.tools.dotc.ast.tpd$TreeProvider.rootTrees(tpd.scala:1299)
at dotty.tools.dotc.ast.tpd$TreeProvider.rootTrees$(tpd.scala:1288)
at dotty.tools.dotc.core.tasty.DottyUnpickler.rootTrees(DottyUnpickler.scala:44)
at dotty.tools.dotc.ast.tpd$TreeProvider.tree(tpd.scala:1303)
at dotty.tools.dotc.ast.tpd$TreeProvider.tree$(tpd.scala:1288)
at dotty.tools.dotc.core.tasty.DottyUnpickler.tree(DottyUnpickler.scala:44)
at dotty.tools.dotc.quoted.PickledQuotes$.unpickle(PickledQuotes.scala:274)
at dotty.tools.dotc.quoted.PickledQuotes$.unpickleTerm(PickledQuotes.scala:84)
at scala.quoted.runtime.impl.QuotesImpl.unpickleExprV2(QuotesImpl.scala:3143)
at Meta$package$.f(Meta.scala:7)
at Meta$package$.$init$$$anonfun$1$$anonfun$1$$anonfun$1(Meta.scala:9)
at Meta$package$.$init$$$anonfun$1$$anonfun$1(Meta.scala:9)
at Meta$package$.$init$$$anonfun$1$$anonfun$adapted$1(Meta.scala:9)
at dotty.tools.dotc.quoted.PickledQuotes$$anon$1.transform(PickledQuotes.scala:110)
at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1538)
at dotty.tools.dotc.quoted.PickledQuotes$$anon$1.transform(PickledQuotes.scala:135)
at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transformBlock(Trees.scala:1603)
at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1518)
at dotty.tools.dotc.quoted.PickledQuotes$$anon$1.transform(PickledQuotes.scala:135)
at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1570)
at dotty.tools.dotc.quoted.PickledQuotes$$anon$1.transform(PickledQuotes.scala:135)
at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform$$anonfun$1(Trees.scala:1605)
at scala.collection.immutable.List.mapConserve(List.scala:472)
at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1605)
at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transformStats(Trees.scala:1601)
at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transformBlock(Trees.scala:1603)
at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1518)
at dotty.tools.dotc.quoted.PickledQuotes$$anon$1.transform(PickledQuotes.scala:135)
at dotty.tools.dotc.quoted.PickledQuotes$.spliceTerms(PickledQuotes.scala:152)
at dotty.tools.dotc.quoted.PickledQuotes$.unpickleTerm(PickledQuotes.scala:88)
at scala.quoted.runtime.impl.QuotesImpl.unpickleExprV2(QuotesImpl.scala:3143)
at Meta$package$.$init$$$anonfun$1(Meta.scala:9)
at scala.quoted.staging.QuoteCompiler$QuotedFrontend.runOn$$anonfun$1(QuoteCompiler.scala:83)
at scala.collection.immutable.List.flatMap(List.scala:293)
at scala.quoted.staging.QuoteCompiler$QuotedFrontend.runOn(QuoteCompiler.scala:98)
at dotty.tools.dotc.Run.runPhases$1$$anonfun$1(Run.scala:246)
at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1321)
at dotty.tools.dotc.Run.runPhases$1(Run.scala:262)
at dotty.tools.dotc.Run.compileUnits$$anonfun$1(Run.scala:270)
at dotty.tools.dotc.Run.compileUnits$$anonfun$adapted$1(Run.scala:279)
at dotty.tools.dotc.util.Stats$.maybeMonitored(Stats.scala:67)
at dotty.tools.dotc.Run.compileUnits(Run.scala:279)
at dotty.tools.dotc.Run.compileUnits(Run.scala:200)
at scala.quoted.staging.QuoteCompiler$ExprRun.compileExpr(QuoteCompiler.scala:118)
at scala.quoted.staging.QuoteDriver.run(QuoteDriver.scala:43)
at scala.quoted.staging.Compiler$$anon$1.run(Compiler.scala:38)
at scala.quoted.staging.package$.run(staging.scala:19)
at Meta$package$.(Meta.scala:9)
... 66 elided

Expectation

...for this not to happen.

[hamzaremmal]

Compiling the file below (i19170.scala) will emit a valid warning

import scala.quoted.*
given staging.Compiler = staging.Compiler.make(getClass.getClassLoader)
class A(i: Int)
def f(i: Expr[Int])(using Quotes): Expr[A] = { '{ new A($i) } }
val g: Int => A = staging.run { '{ (i: Int) => ${ f('{i}) } } }
@main def run = g

Full stack trace when running scala i19170.scala:

-- [E181] Potential Issue Warning: /Users/hamzaremmal/Desktop/LAMP/dotty/i19170.scala:3:47
3 |given staging.Compiler = staging.Compiler.make(getClass.getClassLoader)
  |                                               ^^^^^^^^
  |                         Suspicious top-level unqualified call to getClass
  |
  | longer explanation available when compiling with `-explain`
1 warning found
java.lang.ExceptionInInitializerError
	at run.main(i19170.scala:11)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at dotty.tools.scripting.ScriptingDriver.compileAndRun(ScriptingDriver.scala:33)
	at dotty.tools.scripting.Main$.process(Main.scala:45)
	at dotty.tools.MainGenericRunner$.run$1(MainGenericRunner.scala:250)
	at dotty.tools.MainGenericRunner$.process(MainGenericRunner.scala:270)
	at dotty.tools.MainGenericRunner$.main(MainGenericRunner.scala:281)
	at dotty.tools.MainGenericRunner.main(MainGenericRunner.scala)
Caused by: dotty.tools.dotc.reporting.UnhandledError: undefined: new A # -1: TermRef(TypeRef(ThisType(TypeRef(ThisType(TypeRef(NoPrefix,module class <root>)),module class <empty>)),A),<init>) at quotedFrontend
	at dotty.tools.dotc.reporting.ThrowingReporter.doReport(ThrowingReporter.scala:14)
	at dotty.tools.dotc.reporting.Reporter.issueUnconfigured(Reporter.scala:158)
	at dotty.tools.dotc.reporting.Reporter.go$1(Reporter.scala:181)
	at dotty.tools.dotc.reporting.Reporter.issueIfNotSuppressed(Reporter.scala:200)
	at dotty.tools.dotc.reporting.Reporter.report(Reporter.scala:203)
	at dotty.tools.dotc.report$.error(report.scala:68)
	at dotty.tools.dotc.typer.ErrorReporting$.errorType(ErrorReporting.scala:31)
	at dotty.tools.dotc.typer.TypeAssigner.assignType(TypeAssigner.scala:298)
	at dotty.tools.dotc.typer.TypeAssigner.assignType$(TypeAssigner.scala:16)
	at dotty.tools.dotc.typer.Typer.assignType(Typer.scala:116)
	at dotty.tools.dotc.ast.tpd$.Apply(tpd.scala:49)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readLengthTree$1(TreeUnpickler.scala:1304)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readTree(TreeUnpickler.scala:1468)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readLengthTree$1(TreeUnpickler.scala:1333)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readTree(TreeUnpickler.scala:1468)
	at dotty.tools.dotc.core.tasty.TreeUnpickler.unpickle(TreeUnpickler.scala:117)
	at dotty.tools.dotc.core.tasty.DottyUnpickler.computeRootTrees(DottyUnpickler.scala:62)
	at dotty.tools.dotc.ast.tpd$TreeProvider.rootTrees(tpd.scala:1299)
	at dotty.tools.dotc.ast.tpd$TreeProvider.rootTrees$(tpd.scala:1288)
	at dotty.tools.dotc.core.tasty.DottyUnpickler.rootTrees(DottyUnpickler.scala:44)
	at dotty.tools.dotc.ast.tpd$TreeProvider.tree(tpd.scala:1303)
	at dotty.tools.dotc.ast.tpd$TreeProvider.tree$(tpd.scala:1288)
	at dotty.tools.dotc.core.tasty.DottyUnpickler.tree(DottyUnpickler.scala:44)
	at dotty.tools.dotc.quoted.PickledQuotes$.unpickle(PickledQuotes.scala:274)
	at dotty.tools.dotc.quoted.PickledQuotes$.unpickleTerm(PickledQuotes.scala:84)
	at scala.quoted.runtime.impl.QuotesImpl.unpickleExprV2(QuotesImpl.scala:3143)
	at i19170$package$.f(i19170.scala:7)
	at i19170$package$.$init$$$anonfun$1$$anonfun$1$$anonfun$1(i19170.scala:9)
	at i19170$package$.$init$$$anonfun$1$$anonfun$1(i19170.scala:9)
	at i19170$package$.$init$$$anonfun$1$$anonfun$adapted$1(i19170.scala:9)
	at dotty.tools.dotc.quoted.PickledQuotes$$anon$1.transform(PickledQuotes.scala:110)
	at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1538)
	at dotty.tools.dotc.quoted.PickledQuotes$$anon$1.transform(PickledQuotes.scala:135)
	at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transformBlock(Trees.scala:1603)
	at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1518)
	at dotty.tools.dotc.quoted.PickledQuotes$$anon$1.transform(PickledQuotes.scala:135)
	at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1570)
	at dotty.tools.dotc.quoted.PickledQuotes$$anon$1.transform(PickledQuotes.scala:135)
	at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform$$anonfun$1(Trees.scala:1605)
	at scala.collection.immutable.List.mapConserve(List.scala:472)
	at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1605)
	at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transformStats(Trees.scala:1601)
	at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transformBlock(Trees.scala:1603)
	at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1518)
	at dotty.tools.dotc.quoted.PickledQuotes$$anon$1.transform(PickledQuotes.scala:135)
	at dotty.tools.dotc.quoted.PickledQuotes$.spliceTerms(PickledQuotes.scala:152)
	at dotty.tools.dotc.quoted.PickledQuotes$.unpickleTerm(PickledQuotes.scala:88)
	at scala.quoted.runtime.impl.QuotesImpl.unpickleExprV2(QuotesImpl.scala:3143)
	at i19170$package$.$init$$$anonfun$1(i19170.scala:9)
	at scala.quoted.staging.QuoteCompiler$QuotedFrontend.runOn$$anonfun$1(QuoteCompiler.scala:83)
	at scala.collection.immutable.List.flatMap(List.scala:293)
	at scala.quoted.staging.QuoteCompiler$QuotedFrontend.runOn(QuoteCompiler.scala:98)
	at dotty.tools.dotc.Run.runPhases$1$$anonfun$1(Run.scala:246)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1321)
	at dotty.tools.dotc.Run.runPhases$1(Run.scala:262)
	at dotty.tools.dotc.Run.compileUnits$$anonfun$1(Run.scala:270)
	at dotty.tools.dotc.Run.compileUnits$$anonfun$adapted$1(Run.scala:279)
	at dotty.tools.dotc.util.Stats$.maybeMonitored(Stats.scala:67)
	at dotty.tools.dotc.Run.compileUnits(Run.scala:279)
	at dotty.tools.dotc.Run.compileUnits(Run.scala:200)
	at scala.quoted.staging.QuoteCompiler$ExprRun.compileExpr(QuoteCompiler.scala:118)
	at scala.quoted.staging.QuoteDriver.run(QuoteDriver.scala:43)
	at scala.quoted.staging.Compiler$$anon$1.run(Compiler.scala:38)
	at scala.quoted.staging.package$.run(staging.scala:19)
	at i19170$package$.<clinit>(i19170.scala:9)
	... 11 more

[hamzaremmal]

Running it on the main branch (d96e9e4) gives a different output :

-- [E006] Not Found Error: i19170.scala:3:6 ------------------------------------
3 |given staging.Compiler = staging.Compiler.make(getClass.getClassLoader)
  |      ^^^^^^^
  |      Not found: staging - did you mean String?
  |
  | longer explanation available when compiling with `-explain`
-- [E006] Not Found Error: i19170.scala:3:25 -----------------------------------
3 |given staging.Compiler = staging.Compiler.make(getClass.getClassLoader)
  |                         ^^^^^^^
  |                         Not found: staging - did you mean String?
  |
  | longer explanation available when compiling with `-explain`
-- [E006] Not Found Error: i19170.scala:9:18 -----------------------------------
9 |val g: Int => A = staging.run { '{ (i: Int) => ${ f('{i}) } } }
  |                  ^^^^^^^
  |                  Not found: staging - did you mean String?
  |
  | longer explanation available when compiling with `-explain`
3 errors found

[nicolasstucki]

@hamzaremmal you are missing the dependency on the staging jar. You should use tests/pos-staging to write tests with that dependency .

You can also use the -with-compiler flag.

[christophkoch]

Sorry, my code wasn't minimal. This smaller snippet still causes the crash:

import scala.quoted.*
given staging.Compiler = staging.Compiler.make(getClass.getClassLoader)

class A
val g: A = staging.run { '{ new A } }

Also, I would be curious to learn from the experts how to get rid of the warning about the "given staging.Compiler = ..." line, which Hamza also reported. This line is boilerplate I copied in from the latest online documentation, which may be outdated.

[hamzaremmal]

Also, I would be curious to learn from the experts how to get rid of the warning about the "given staging.Compiler = ..." line, which Hamza also reported.

The warning comes from the fact that you're calling a method from the top level, in this case getClass, and that this function call might end up calling the wrong method. That's why the message of the warning contains the following "Suspicious top-level unqualified call to getClass". The best way to get rid of this warning is to qualify the call to getClass, eg. Predef.getClass

@nicolasstucki Do you think this can be solved during a Spree ?

[christophkoch]

Thanks a lot, Hamza! It would be good to fix this in the documentation, e.g. at
https://docs.scala-lang.org/scala3/reference/metaprogramming/staging.html

[nicolasstucki]

The warning comes from the fact that you're calling a method from the top level, in this case getClass, and that this function call might end up calling the wrong method. That's why the message of the warning contains the following "Suspicious top-level unqualified call to getClass". The best way to get rid of this warning is to qualify the call to getClass, eg. Predef.getClass

This points to the actual issue with this code, but it does not make the solution evident. In the past, when we called getClass in a top-level method we used to call this.getClass where this was on the synthetic package object. Now we get a Predef.getClass.

What we need to pass to staging.Compiler.make is the classloader that loaded the application. Some tools (scala, SBT, or others) load the standard library before the application is loaded in a parent classloader. If we use this classloader, the runtime staging compiler will not know how to load the classes of the application.

This seems to be the simplest workaround for top-level definitions

given staging.Compiler =
  object Dummy
  staging.Compiler.make(Dummy.getClass.getClassLoader)

We should update the documentation using this version, unless we find a simpler safe incantation for this classloader.

[christophkoch]

Thanks a lot Nicolas. If I do this in the Repl, I get

dotty.tools.dotc.MissingCoreLibraryException: Could not find package scala from compiler core libraries.
Make sure the compiler core libraries are on the classpath.

Is there an easy way to fix this?