Getting Position of Tree fails execution

[BarkingBad]

Compiler version

3.1.0-RC1-bin-20210820-68044a6-NIGHTLY

When using traverseTree from Quotes TreeTraverser trait I check the position of each tree I visit. For some reason accessing the pos property, the execution fails. The stacktrace is leading to the assertion of the Span existence. The stacktrace is as follow:

Stacktrace

[error] Exception in thread "main" java.lang.AssertionError: assertion failed
[error]         at scala.runtime.Scala3RunTime$.assertFailed(Scala3RunTime.scala:11)
[error]         at dotty.tools.dotc.util.Spans$Span$.start$extension(Spans.scala:44)
[error]         at dotty.tools.dotc.util.SourcePosition.start(SourcePosition.scala:50)
[error]         at dotty.tools.dotc.util.SourcePosition.startLine(SourcePosition.scala:51)
[error]         at scala.quoted.runtime.impl.QuotesImpl$reflect$PositionMethods$.startLine(QuotesImpl.scala:2727)
[error]         at scala.quoted.runtime.impl.QuotesImpl$reflect$PositionMethods$.startLine(QuotesImpl.scala:2727)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.traverseTree(StacktracesInspector.scala:73)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.foldTree(StacktracesInspector.scala:78)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.foldTree(StacktracesInspector.scala:78)
[error]         at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldOverTree(Quotes.scala:4329)
[error]         at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldOverTree$(Quotes.scala:4247)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.foldOverTree(StacktracesInspector.scala:56)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.traverseTreeChildren(StacktracesInspector.scala:80)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.traverseTree(StacktracesInspector.scala:74)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.foldTree(StacktracesInspector.scala:78)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.foldTree(StacktracesInspector.scala:78)
[error]         at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldOverTree(Quotes.scala:4304)
[error]         at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldOverTree$(Quotes.scala:4247)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.foldOverTree(StacktracesInspector.scala:56)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.traverseTreeChildren(StacktracesInspector.scala:80)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.traverseTree(StacktracesInspector.scala:74)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.foldTree(StacktracesInspector.scala:78)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.foldTree(StacktracesInspector.scala:78)
[error]         at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldTrees$$anonfun$1(Quotes.scala:4252)
[error]         at scala.collection.LinearSeqOps.foldLeft(LinearSeq.scala:169)
[error]         at scala.collection.LinearSeqOps.foldLeft$(LinearSeq.scala:165)
[error]         at scala.collection.immutable.List.foldLeft(List.scala:79)
[error]         at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldTrees(Quotes.scala:4252)
[error]         at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldTrees$(Quotes.scala:4247)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.foldTrees(StacktracesInspector.scala:56)
[error]         at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldOverTree(Quotes.scala:4307)
[error]         at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldOverTree$(Quotes.scala:4247)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.foldOverTree(StacktracesInspector.scala:56)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.traverseTreeChildren(StacktracesInspector.scala:80)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.traverseTree(StacktracesInspector.scala:74)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.foldTree(StacktracesInspector.scala:78)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.foldTree(StacktracesInspector.scala:78)
[error]         at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldTrees$$anonfun$1(Quotes.scala:4252)
[error]         at scala.collection.LinearSeqOps.foldLeft(LinearSeq.scala:169)
[error]         at scala.collection.LinearSeqOps.foldLeft$(LinearSeq.scala:165)
[error]         at scala.collection.immutable.List.foldLeft(List.scala:79)
[error]         at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldTrees(Quotes.scala:4252)
[error]         at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldTrees$(Quotes.scala:4247)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.foldTrees(StacktracesInspector.scala:56)
[error]         at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldOverTree(Quotes.scala:4313)
[error]         at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldOverTree$(Quotes.scala:4247)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.foldOverTree(StacktracesInspector.scala:56)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.traverseTreeChildren(StacktracesInspector.scala:80)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.traverseTree(StacktracesInspector.scala:74)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.foldTree(StacktracesInspector.scala:78)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.foldTree(StacktracesInspector.scala:78)
[error]         at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldTrees$$anonfun$1(Quotes.scala:4252)
[error]         at scala.collection.LinearSeqOps.foldLeft(LinearSeq.scala:169)
[error]         at scala.collection.LinearSeqOps.foldLeft$(LinearSeq.scala:165)
[error]         at scala.collection.immutable.List.foldLeft(List.scala:79)
[error]         at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldTrees(Quotes.scala:4252)
[error]         at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldTrees$(Quotes.scala:4247)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.foldTrees(StacktracesInspector.scala:56)
[error]         at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldOverTree(Quotes.scala:4313)
[error]         at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldOverTree$(Quotes.scala:4247)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.foldOverTree(StacktracesInspector.scala:56)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.traverseTreeChildren(StacktracesInspector.scala:80)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.traverseTree(StacktracesInspector.scala:74)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.foldTree(StacktracesInspector.scala:78)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.foldTree(StacktracesInspector.scala:78)
[error]         at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldTrees$$anonfun$1(Quotes.scala:4252)
[error]         at scala.collection.LinearSeqOps.foldLeft(LinearSeq.scala:169)
[error]         at scala.collection.LinearSeqOps.foldLeft$(LinearSeq.scala:165)
[error]         at scala.collection.immutable.List.foldLeft(List.scala:79)
[error]         at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldTrees(Quotes.scala:4252)
[error]         at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldTrees$(Quotes.scala:4247)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.foldTrees(StacktracesInspector.scala:56)
[error]         at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldOverTree(Quotes.scala:4313)
[error]         at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldOverTree$(Quotes.scala:4247)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.foldOverTree(StacktracesInspector.scala:56)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.traverseTreeChildren(StacktracesInspector.scala:80)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$Traverser$1.traverseTree(StacktracesInspector.scala:74)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector.inspect$$anonfun$1(StacktracesInspector.scala:114)
[error]         at scala.collection.immutable.List.foreach(List.scala:333)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector.inspect(StacktracesInspector.scala:119)
[error]         at scala.tasty.inspector.TastyInspector$TastyInspectorPhase$1.runOn(TastyInspector.scala:67)
[error]         at dotty.tools.dotc.Run.runPhases$4$$anonfun$4(Run.scala:205)
[error]         at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
[error]         at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
[error]         at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1323)
[error]         at dotty.tools.dotc.Run.runPhases$5(Run.scala:216)
[error]         at dotty.tools.dotc.Run.compileUnits$$anonfun$1(Run.scala:224)
[error]         at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
[error]         at dotty.tools.dotc.util.Stats$.maybeMonitored(Stats.scala:68)
[error]         at dotty.tools.dotc.Run.compileUnits(Run.scala:231)
[error]         at dotty.tools.dotc.Run.compileUnits(Run.scala:172)
[error]         at dotty.tools.dotc.fromtasty.TASTYRun.compile(TASTYRun.scala:12)
[error]         at dotty.tools.dotc.Driver.doCompile(Driver.scala:39)
[error]         at dotty.tools.dotc.Driver.process(Driver.scala:199)
[error]         at dotty.tools.dotc.Driver.process(Driver.scala:167)
[error]         at dotty.tools.dotc.Driver.process(Driver.scala:179)
[error]         at scala.tasty.inspector.TastyInspector$.inspectFiles(TastyInspector.scala:106)
[error]         at scala.tasty.inspector.TastyInspector$.inspectAllTastyFiles(TastyInspector.scala:52)
[error]         at org.virtuslab.stacktraces.core.StacktracesInspector$.inspectStackTrace(StacktracesInspector.scala:28)
[error]         at org.virtuslab.stacktraces.core.Stacktraces$.convertToPrettyStackTrace(Stacktraces.scala:26)
[error]         at org.virtuslab.stacktraces.TestExecutor$package$.convertToPrettyStackTraceWithStdlib(TestExecutor.scala:17)
[error]         at org.virtuslab.stacktraces.TestExecutor$.executeTest(TestExecutor.scala:13)
[error]         at org.virtuslab.stacktraces.tests.BasicTests$.nestedLambdas(BasicTest.scala:37)
[error]         at org.virtuslab.stacktraces.tests.nestedLambdas.main(BasicTest.scala:27)
[error] Nonzero exit code returned from runner: 1

The trees that were accessed in the traverser callback that cause failure

Failing trees

[info] TypeTree[TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),object scala),Nothing)]
[info] TypeTree[TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),object scala),Any)]
[info] TypeTree[TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),object scala),Nothing)]
[info] TypeTree[TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),object scala),Any)]
[info] TypeTree[TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),object scala),Nothing)]
[info] TypeTree[TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),object scala),Any)]
[info] TypeTree[TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),object scala),Nothing)]
[info] TypeTree[TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),object scala),Any)]
[info] TypeTree[TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),object scala),Nothing)]
[info] TypeTree[HKTypeLambda(List(_$1), List(TypeBounds(TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),object scala),Nothing),TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),object scala),Any))), TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),object scala),Any), List())]
[info] TypeTree[TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),object scala),Nothing)]
[info] TypeTree[TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),object scala),Any)]
[info] TypeTree[TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),object scala),Nothing)]
[info] TypeTree[TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),object scala),Any)]
[info] TypeTree[TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),object scala),Nothing)]
[info] TypeTree[HKTypeLambda(List(_$1), List(TypeBounds(TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),object scala),Nothing),TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),object scala),Any))), TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),object scala),Any), List())]
[info] TypeTree[TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),object scala),Nothing)]
[info] TypeTree[TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),object scala),Any)]

There used to be the exists function in Position to check whether the Span exists but IIRC was removed because accessing pos should be safe now. I can provide a sample repo if necessary.

[soronpo]

Please try sticking to the issue format and providing a reproducing method with minimized code.

[nicolasstucki]

How can this be reproduced?

[BarkingBad]

Sorry for not responding, I was on vacation. Here is the snippet:

import scala.quoted.*
import scala.tasty.inspector.*

class MyInspector extends Inspector:

  override def inspect(using q: Quotes)(tastys: List[Tasty[quotes.type]]): Unit =
    import q.reflect.*
    class Traverser extends TreeTraverser:
      override def traverseTree(tree: Tree)(owner: Symbol) = 
        if tree.pos.startLine < 100 then 
          super.traverseTree(tree)(owner)
    end Traverser

    val traverser = new Traverser
    tastys.foreach { tasty =>
      traverser.traverseTree(tasty.ast)(tasty.ast.symbol)
    }
    
def list_urls(cl: ClassLoader): Array[java.net.URL] = cl match {
  case null => Array()
  case u: java.net.URLClassLoader => u.getURLs() ++ list_urls(cl.getParent)
  case _ => list_urls(cl.getParent)
}

@main def main =
  val urls = list_urls(getClass.getClassLoader).distinct
  val inspector = new MyInspector
  TastyInspector.inspectAllTastyFiles(Nil, urls.map(x => java.nio.file.Paths.get(x.toURI).toString).toList, Nil)(inspector)

It is not that simple because we need to get some tasties to run them on, but this should get the classpath and run inspector on top of that. The crucial part is accessing tree.pos... that makes the whole execution fail.

[nicolasstucki]

@BarkingBad could you try to make it a test in tests/run-custom-args/tasty-inspector?

[nicolasstucki]

Ok, could reproduce it with tests/run-custom-args/tasty-inspector/i13352.scala

import scala.quoted.*
import scala.tasty.inspector.*

@main def Test = {
  // Artefact of the current test infrastructure
  // TODO improve infrastructure to avoid needing this code on each test
  val classpath = dotty.tools.dotc.util.ClasspathFromClassloader(this.getClass.getClassLoader).split(java.io.File.pathSeparator).find(_.contains("runWithCompiler")).get
  val allTastyFiles = dotty.tools.io.Path(classpath).walkFilter(_.extension == "tasty").map(_.toString).toList
  val tastyFiles = allTastyFiles.filter(_.contains("CanEqual"))

  TastyInspector.inspectTastyFiles(tastyFiles)(new MyInspector)
}

class MyInspector extends Inspector:

  override def inspect(using Quotes)(tastys: List[Tasty[quotes.type]]): Unit =
    import quotes.reflect.*
    class Traverser extends TreeTraverser:
      override def traverseTree(tree: Tree)(owner: Symbol) =
        if tree.pos.startLine < 100 then
          super.traverseTree(tree)(owner)
    end Traverser

    val traverser = new Traverser
    tastys.foreach { tasty =>
      traverser.traverseTree(tasty.ast)(tasty.ast.symbol)
    }


import annotation.implicitNotFound
import scala.collection.{Seq, Set}

/** A marker trait indicating that values of type `L` can be compared to values of type `R`. */
@implicitNotFound("Values of types ${L} and ${R} cannot be compared with == or !=")
sealed trait CanEqual[-L, -R]

/** Companion object containing a few universally known `CanEqual` instances.
 *  CanEqual instances involving primitive types or the Null type are handled directly in
 *  the compiler (see Implicits.synthesizedCanEqual), so they are not included here.
 */
object CanEqual {
  /** A universal `CanEqual` instance. */
  object derived extends CanEqual[Any, Any]

  /** A fall-back instance to compare values of any types.
   *  Even though this method is not declared as given, the compiler will
   *  synthesize implicit arguments as solutions to `CanEqual[T, U]` queries if
   *  the rules of multiversal equality require it.
   */
  def canEqualAny[L, R]: CanEqual[L, R] = derived

  // Instances of `CanEqual` for common Java types
  given canEqualNumber: CanEqual[Number, Number] = derived
  given canEqualString: CanEqual[String, String] = derived

  // The next 6 definitions can go into the companion objects of their corresponding
  // classes. For now they are here in order not to have to touch the
  // source code of these classes
  given canEqualSeqs[T, U](using eq: CanEqual[T, U]): CanEqual[Seq[T], Seq[U]] = derived
  given canEqualSeq[T](using eq: CanEqual[T, T]): CanEqual[Seq[T], Seq[T]] = derived // for `case Nil` in pattern matching

  given canEqualSet[T, U](using eq: CanEqual[T, U]): CanEqual[Set[T], Set[U]] = derived

  given canEqualOptions[T, U](using eq: CanEqual[T, U]): CanEqual[Option[T], Option[U]] = derived
  given canEqualOption[T](using eq: CanEqual[T, T]): CanEqual[Option[T], Option[T]] = derived // for `case None` in pattern matching

  given canEqualEither[L1, R1, L2, R2](
    using eqL: CanEqual[L1, L2], eqR: CanEqual[R1, R2]
  ): CanEqual[Either[L1, R1], Either[L2, R2]] = derived
}

[nicolasstucki]

Minimized to

import scala.quoted.*
import scala.tasty.inspector.*

@main def Test = {
  // Artefact of the current test infrastructure
  // TODO improve infrastructure to avoid needing this code on each test
  val classpath = dotty.tools.dotc.util.ClasspathFromClassloader(this.getClass.getClassLoader).split(java.io.File.pathSeparator).find(_.contains("runWithCompiler")).get
  val allTastyFiles = dotty.tools.io.Path(classpath).walkFilter(_.extension == "tasty").map(_.toString).toList
  val tastyFiles = allTastyFiles.filter(_.contains("CanEqual2"))

  TastyInspector.inspectTastyFiles(tastyFiles)(new MyInspector)
}

class MyInspector extends Inspector:

  override def inspect(using Quotes)(tastys: List[Tasty[quotes.type]]): Unit =
    import quotes.reflect.*
    class Traverser extends TreeTraverser:
      override def traverseTree(tree: Tree)(owner: Symbol) =
        println(tree)
        println(" ")
        if tree.pos.startLine < 100 then
          super.traverseTree(tree)(owner)
    end Traverser

    val traverser = new Traverser
    tastys.foreach { tasty =>
      traverser.traverseTree(tasty.ast)(tasty.ast.symbol)
    }


class CanEqual2[T]

[BarkingBad]

I was about to move it to the tests but you were faster, thanks for your input