`compiletime.testing.typeCheckErrors` doesn't report macro errors

[japgolly]

Compiler version

3.0.0-RC1

Minimized code

a.scala:

object X:
  import scala.quoted.*

  inline def f: Int = ${ g }
  private def g(using Quotes): Expr[Int] = quotes.reflect.report.throwError("nope")

b.scala:

def test() =
  println(scala.compiletime.testing.typeCheckErrors("X.f")) // prints List()
  X.f // compilation error as expected

Output

• The typeCheckErrors line prints List()
• The X.f line fails to compile (as expected)

Expectation

typeCheckErrors should return List("nope") because that's the compilation error emitted by X.f.

Context

I came across this because I'm adding Scala 3 support to a Scala 2 library with macros. There are unit tests and ensure that the macros fail in certain cases and with Scala 2, it was enough to wrap the code in uTest's compileError(). It works fine for Scala 2 but when I try to run the shared tests against the Scala 3 code, uTest's compileError() reports failed-macro-calls as having no errors.

Looking at how compileError in implemented in uTest 0.7.7 / Scala 3.0.0-RC1, it uses typeCheckErrors. If typeCheckErrors isn't supposed to catch macro algorithms, then is there an alternative? There needs to be some kind of way because metaprogramming needs to be tested for all the same reasons that direct programming needs testing.

[smarter]

Have you tried with a nightly? I believe #11656 fixed that.

[japgolly]

You're right! Just tried with a nightly and it works as expected. Thanks!