Invalid code passes scala.compiletime.testing.typeChecks with Generic Number Literals

[benhutchison]

In Scala 3-RC1, Im seeing some apparently flaky behavior from scala.compiletime.testing.typeChecks()

I have a test for compile-time checking of Generic Number Literals, almost exactly like Even in the dotty test codebase.
https://github.com/lampepfl/dotty/blob/ed5ecac22b2/tests/neg-macros/GenericNumLits/Test_2.scala#L5

The difference is Im using typeChecks in my unit test. Invalid number literals are reported as successfully type checking. If I take the exact same code outside of a typeCheck invocation, it fails to compile as expected.

Compiler version

3.0.0-RC1

Minimized code

Change line https://github.com/lampepfl/dotty/blob/ed5ecac22b2/tests/neg-macros/GenericNumLits/Test_2.scala#L5

println(scala.compiletime.testing.typeChecks("val e2: Even = 123"))

Output

true

Expectation

false

[nicolasstucki]

The issue is that the error is not a type-checking error. It is an error that happens when we inline. You may use transparent if you want this error to be a type-checking error.

[odersky]

I think this line in Even_1 needs to be changed to transparent.

override inline def fromDigits(digits: String) = ${`

[benhutchison]

Actually, when I try adding transparent to my example, valid Even cases are no longer accepted by the macro.

If transparent ought to work, would be great to see it working in the dotty codebase so I can confirm Im using it correctly.