`-coverage-out` generates coverage information for macro code coming from referenced libraries

[arixmkii]

Compiler version

3.2.0-RC1

If you're not sure what version you're using, run print scalaVersion from sbt
(if you're running scalac manually, use scalac -version instead).

Minimized code

Macro.scala

package excludeme

import scala.quoted.*

object Macro {

  inline def toUpperS(s: String): String = ${ Macro.toUpperSImpl('s) }

  def toUpperSImpl(s: Expr[String])(using q: Quotes): Expr[String] = '{ ($s.toUpperCase) } // will add coverage info
}

Main.scala

import excludeme.*

@main def main: Unit =
  println(Macro.toUpperS("test"))

Minimized code (using third party library)

Keeping this for the sake of history how it was originally reported.

import io.circe.*
import io.circe.generic.semiauto.*
import io.circe.syntax.*

implicit val basicCodec: Codec[Tuple1[String]] = deriveCodec[Tuple1[String]]

@main def main: Unit =
  println(Tuple1("value").asJson.noSpaces)

Full project is here https://github.com/arixmkii/macro_cov_3.2.0-RC1
I will think about a better minimal example, because circe-generic is a heavy one. May be with 2 source files and using one with macro as pre-compiled class.

Output

scoverage.coverage file contains records of

modules/core/shared/src/main/scala-3/io/circe/Derivation.scala

Expectation

No coverage data for third party libraries as source might not be provided for them.

Originally posted here scoverage/scalac-scoverage-plugin#474 I think that workarounds could be added to report generation to handle such cases (probably giving out a warning by default), but I'm not sure if this is the right approach for dealing with macros from referenced libraries.

[arixmkii]

First try on minimization:

Macro definitions:
Macro.scala

package excludeme

sealed abstract class MyToken extends Product with Serializable {}

final case class MyTokenS(s: String) extends MyToken

object MyToken {
  def fromString(s: String): MyToken = MyTokenS(s)
}

import scala.quoted.*

object Macro {
  inline def makeToken(s: String): MyToken = ${ Macro.makeTokenImpl('s) }

  def makeTokenImpl(s: Expr[String])(using q: Quotes): Expr[MyToken] = {
    '{ MyToken.fromString($s) } // will add coverage info
//    '{ MyTokenS.apply($s) } // will not add coverage info
  }
}

Prorgam:
Main.scala

import excludeme.*

@main def main: Unit =
  println(Macro.makeToken("test"))

Compile wth -coverage-out enabled and run from sbt.

Scoverage data will have this erroneous record.

0
src/main/scala/Macro.scala
<empty>
Main$package$
Object
<empty>.Main$package$
main
681
709
25
fromString
Apply
false
0
false
MyToken.fromString($strExpr)

Trying to generate coverage data for fromString on MyToken companion.

Expected result - no such record.

If I change last line of a macro to

    '{ MyTokenS.apply($s) }

Directly calling apply on a case class - there will be no coverage info added (as expected).

[arixmkii]

As minimal as I can get it.
Macro.scala

package excludeme

import scala.quoted.*

object Macro {

  inline def toUpperS(s: String): String = ${ Macro.toUpperSImpl('s) }

  def toUpperSImpl(s: Expr[String])(using q: Quotes): Expr[String] = '{ ($s.toUpperCase) } // will add coverage info
}

Main.scala

import excludeme.*

@main def main: Unit =
  println(Macro.toUpperS("test"))

Will update the description.

[Kordyjan]

Thanks for the minimalization!

[nicolasstucki]

What is work in the minimized version? What is the expected output? Which command should I use to reproduce the issue?

[arixmkii]

# compile Macro
./scala3-3.2.0-RC1/bin/scalac Macro.scala
# remove source (for clean experiment) and keep only compiled
rm Macro.scala
# compile program with coverage
./scala3-3.2.0-RC1/bin/scalac -coverage-out:. Main.scala
# check coverage data
cat scoverage.coverage | grep -A14 Macro.scala

Outputs

Macro.scala
<empty>
Main$package$
Object
<empty>.Main$package$
main
203
217
8
toUpperCase
Apply
false
0
false

But we removed this source and used only compiled module. So, there should be no coverage info for this. Same is true for macros coming from third party libraries (which then breaks coverage report, because sources can't be found).

I expect that there will be no coverage info generated from code instantiated via macro.

[nicolasstucki]

If I understand correctly you are saying that if we have code for a source that we do not have we should not emit the info in the coverage file.

Wouldn't it be more resilient if the reporting part is resilient to missing files? We also have the situation where the file is deleted after the code is compiled.