Unresolved symbols when pickling quotes: by-name parameters, underlyingArgument and unintuitive error message

[anatoliykmetyuk]

Macros:

import scala.quoted._

inline def inner(exprs: Any): Any = ${innerImpl('exprs)}
def innerImpl(exprs: Expr[Any]) given QuoteContext: Expr[Any] =
  '{$exprs ; ()}

inline def outer(expr: => Any): Any = ${outerImpl('expr)}
def outerImpl(body: Expr[Any]) given (ctx: QuoteContext): Expr[Any] = {
  import ctx.tasty._
  body.unseal.underlyingArgument.seal
}

Test:

val x = outer(inner(???))

Crash

exception occurred while compiling ../pg-dotc/Main.scala
java.lang.AssertionError: assertion failed: unresolved symbols: value exprs(line 0) when pickling ../pg-dotc/Main.scala while compiling ../pg-dotc/Main.scala
Exception in thread "main" java.lang.AssertionError: assertion failed: unresolved symbols: value exprs(line 0) when pickling ../pg-dotc/Main.scala
	at dotty.DottyPredef$.assertFail(DottyPredef.scala:16)
	at dotty.tools.dotc.core.tasty.TreePickler.pickle(TreePickler.scala:695)
	at dotty.tools.dotc.transform.Pickler.run$$anonfun$10$$anonfun$8(Pickler.scala:60)
	at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.immutable.List.foreach(List.scala:392)
	at dotty.tools.dotc.transform.Pickler.run$$anonfun$2(Pickler.scala:83)
	at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.immutable.List.foreach(List.scala:392)
	at dotty.tools.dotc.transform.Pickler.run(Pickler.scala:83)
	at dotty.tools.dotc.core.Phases$Phase.runOn$$anonfun$1(Phases.scala:316)
	at scala.collection.immutable.List.map(List.scala:286)
	at dotty.tools.dotc.core.Phases$Phase.runOn(Phases.scala:318)
	at dotty.tools.dotc.transform.Pickler.runOn(Pickler.scala:87)
	at dotty.tools.dotc.Run.runPhases$4$$anonfun$4(Run.scala:158)
	at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36)
	at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33)
	at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:198)
	at dotty.tools.dotc.Run.runPhases$5(Run.scala:170)
	at dotty.tools.dotc.Run.compileUnits$$anonfun$1(Run.scala:178)
	at dotty.runtime.function.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
	at dotty.tools.dotc.util.Stats$.maybeMonitored(Stats.scala:65)
	at dotty.tools.dotc.Run.compileUnits(Run.scala:185)
	at dotty.tools.dotc.Run.compileSources(Run.scala:120)
	at dotty.tools.dotc.Run.compile(Run.scala:104)
	at dotty.tools.dotc.Driver.doCompile(Driver.scala:35)
	at dotty.tools.dotc.Driver.process(Driver.scala:180)
	at dotty.tools.dotc.Driver.process(Driver.scala:149)
	at dotty.tools.dotc.Driver.process(Driver.scala:161)
	at dotty.tools.dotc.Driver.main(Driver.scala:188)
	at dotty.tools.dotc.Main.main(Main.scala)
[error] Nonzero exit code returned from runner: 1
[error] (dotty-compiler / Compile / runMain) Nonzero exit code returned from runner: 1
[error] Total time: 8 s, completed Aug 12, 2019 2:07:51 PM

To make it compile, make the parameter to inner by-name:

inline def inner(exprs: => Any): Any = ${innerImpl('exprs)}

In my original use case, it was a mistake for me not to make it by-name. However, due to the obscure crash, I was not able to see that mistake right away.

This is already the second time I encounter unintuitive behaviour of underlyingArgument (don't remember the first time, but it was likely the same thing). I expected it to just return an immediate child node of the callee. However, it seems that it performs a deep traversal and transformation of the callee tree. If you debug outerImpl as follows:

def outerImpl(body: Expr[Any]) given (ctx: QuoteContext): Expr[Any] = {
  import ctx.tasty._
  println(s"Original:\n${body.unseal}")
  println(s"\nUnderlying:\n${body.unseal.underlyingArgument}")
  body.unseal.underlyingArgument.seal
}

Original will be

Inlined(
  EmptyTree,List(

  ),Inlined(
    Apply(
      Ident(
        inner
      ),List(
        Ident(
          ???
        )
      )
    ),List(
      ValDef(
        exprs,TypeTree[TypeRef(
          ThisType(
            TypeRef(
              NoPrefix,module class scala
            )
          ),class Nothing
        )],Ident(
          ???
        )
      )
    ),Typed(
      Inlined(
        Ident(
          Macros$package$
        ),List(

        ),Block(
          List(
            Inlined(
              EmptyTree,List(

              ),Inlined(
                EmptyTree,List(

                ),Ident(
                  exprs
                )
              )
            )
          ),Literal(
            Constant(
              (

              )
            )
          )
        )
      ),Ident(
        Any
      )
    )
  )
)

Underlying will be

Typed(
  Block(
    List(
      Ident(
        exprs
      )
    ),Literal(
      Constant(
        (

        )
      )
    )
  ),Ident(
    Any
  )
)

Notice how the following code is not present in Underlying:

      ValDef(
        exprs,TypeTree[TypeRef(
          ThisType(
            TypeRef(
              NoPrefix,module class scala
            )
          ),class Nothing
        )],Ident(
          ???
        )
      )

Which is the likely cause of the crash.

I believe underlyingArgument should do just that – pill off the "outer layer" of the tree, returning the meaningful child. If the user really wants the current logic, we should make it into a separate method with a name reflecting its deep nature.

/cc @liufengyun since you looked into the issues with unresolved symbols while pickling recently

[nicolasstucki]

We are supposed to also replace exprs with its underlyingArgument. We might be missing that tree when we try to do that. Or maybe not performing the recursive transformation completely.