post-2.12.1 regression: MatchError in scala.reflect.internal.Trees$Template <init>

[SethTisue]

this turned up in the Scala 2.12 community build.

the regression might not be super recent — I suspect (EDIT: confirmed) that the failing projects all had Shapeless as a direct or indirect dependency, so they weren't being built during the time Shapeless was broken because of SI-10154

the three failing projects in e.g. https://scala-ci.typesafe.com/job/scala-2.12.x-integrate-community-build/1264/consoleFull are akka-http, breeze, and monix.

here is one of the stack traces:

[breeze] scala.MatchError: _root_.java.lang.Object {
[breeze]   def <init>()
[breeze] } (of class scala.reflect.internal.Trees$Template)
[breeze] 	at scala.reflect.internal.ReificationSupport$ReificationSupportImpl$SyntacticClassDef$.unapply(ReificationSupport.scala:354)
[breeze] 	at org.scalamacros.paradise.reflect.TreeInfo$ParadiseTreeInfo.loop$1(TreeInfo.scala:31)
[breeze] 	at org.scalamacros.paradise.reflect.TreeInfo$ParadiseTreeInfo.getAnnotationZippers(TreeInfo.scala:117)
[breeze] 	at org.scalamacros.paradise.typechecker.Namers$Namer.finishSymbol(Namers.scala:182)
[breeze] 	at org.scalamacros.paradise.typechecker.Namers$Namer.finishSymbol$(Namers.scala:166)
[breeze] 	at org.scalamacros.paradise.typechecker.Namers$$anon$3.finishSymbol(Namers.scala:13)
[breeze] 	at org.scalamacros.paradise.typechecker.Namers$Namer.dispatch$1(Namers.scala:35)
[breeze] 	at org.scalamacros.paradise.typechecker.Namers$Namer.enterSym(Namers.scala:41)
[breeze] 	at org.scalamacros.paradise.typechecker.Namers$Namer.enterSym$(Namers.scala:23)
[breeze] 	at org.scalamacros.paradise.typechecker.Namers$$anon$3.enterSym(Namers.scala:13)
[breeze] 	at org.scalamacros.paradise.typechecker.AnalyzerPlugins$MacroPlugin$.pluginsEnterSym(AnalyzerPlugins.scala:36)
[breeze] 	at scala.tools.nsc.typechecker.AnalyzerPlugins$$anon$13.custom(AnalyzerPlugins.scala:429)
[breeze] 	at scala.tools.nsc.typechecker.AnalyzerPlugins.$anonfun$invoke$3(AnalyzerPlugins.scala:371)
[breeze] 	at scala.tools.nsc.typechecker.AnalyzerPlugins.invoke(AnalyzerPlugins.scala:371)
[breeze] 	at scala.tools.nsc.typechecker.AnalyzerPlugins.pluginsEnterSym(AnalyzerPlugins.scala:423)
[breeze] 	at scala.tools.nsc.typechecker.AnalyzerPlugins.pluginsEnterSym$(AnalyzerPlugins.scala:421)
[breeze] 	at scala.tools.nsc.Global$$anon$1.pluginsEnterSym(Global.scala:432)
[breeze] 	at scala.tools.nsc.typechecker.Namers$Namer.enterSym(Namers.scala:273)
[breeze] 	at scala.tools.nsc.typechecker.Namers$Namer.$anonfun$enterSyms$1(Namers.scala:504)
[breeze] 	at scala.tools.nsc.typechecker.Namers$Namer.enterSyms(Namers.scala:503)
[breeze] 	at org.scalamacros.paradise.typechecker.Namers$Namer.$anonfun$finishSymbolNotExpandee$1(Namers.scala:221)
[breeze] 	at org.scalamacros.paradise.typechecker.Namers$Namer.savingLock$1(Namers.scala:215)
[breeze] 	at org.scalamacros.paradise.typechecker.Namers$Namer.finishSymbolNotExpandee(Namers.scala:219)
[breeze] 	at org.scalamacros.paradise.typechecker.Namers$Namer.finishSymbolNotExpandee$(Namers.scala:211)
[breeze] 	at org.scalamacros.paradise.typechecker.Namers$$anon$3.finishSymbolNotExpandee(Namers.scala:13)
[breeze] 	at org.scalamacros.paradise.typechecker.Namers$Namer.finishSymbol(Namers.scala:183)
[breeze] 	at org.scalamacros.paradise.typechecker.Namers$Namer.finishSymbol$(Namers.scala:166)
[breeze] 	at org.scalamacros.paradise.typechecker.Namers$$anon$3.finishSymbol(Namers.scala:13)
[breeze] 	at org.scalamacros.paradise.typechecker.Namers$Namer.dispatch$1(Namers.scala:35)
[breeze] 	at org.scalamacros.paradise.typechecker.Namers$Namer.enterSym(Namers.scala:41)
[breeze] 	at org.scalamacros.paradise.typechecker.Namers$Namer.enterSym$(Namers.scala:23)
[breeze] 	at org.scalamacros.paradise.typechecker.Namers$$anon$3.enterSym(Namers.scala:13)
[breeze] 	at org.scalamacros.paradise.typechecker.AnalyzerPlugins$MacroPlugin$.pluginsEnterSym(AnalyzerPlugins.scala:36)
[breeze] 	at scala.tools.nsc.typechecker.AnalyzerPlugins$$anon$13.custom(AnalyzerPlugins.scala:429)
[breeze] 	at scala.tools.nsc.typechecker.AnalyzerPlugins.$anonfun$invoke$3(AnalyzerPlugins.scala:371)
[breeze] 	at scala.tools.nsc.typechecker.AnalyzerPlugins.invoke(AnalyzerPlugins.scala:371)
[breeze] 	at scala.tools.nsc.typechecker.AnalyzerPlugins.pluginsEnterSym(AnalyzerPlugins.scala:423)
[breeze] 	at scala.tools.nsc.typechecker.AnalyzerPlugins.pluginsEnterSym$(AnalyzerPlugins.scala:421)
[breeze] 	at scala.tools.nsc.Global$$anon$1.pluginsEnterSym(Global.scala:432)
[breeze] 	at scala.tools.nsc.typechecker.Namers$Namer.enterSym(Namers.scala:273)
[breeze] 	at scala.tools.nsc.typechecker.Analyzer$namerFactory$$anon$1.apply(Analyzer.scala:43)
[breeze] 	at scala.tools.nsc.Global$GlobalPhase.$anonfun$applyPhase$1(Global.scala:410)
[breeze] 	at scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:403)
[breeze] 	at scala.tools.nsc.Global$GlobalPhase.$anonfun$run$1(Global.scala:374)
[breeze] 	at scala.tools.nsc.Global$GlobalPhase.$anonfun$run$1$adapted(Global.scala:374)
[breeze] 	at scala.collection.Iterator.foreach(Iterator.scala:929)
[breeze] 	at scala.collection.Iterator.foreach$(Iterator.scala:929)
[breeze] 	at scala.collection.AbstractIterator.foreach(Iterator.scala:1406)
[breeze] 	at scala.tools.nsc.Global$GlobalPhase.run(Global.scala:374)
[breeze] 	at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1413)
[breeze] 	at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1398)
[breeze] 	at scala.tools.nsc.Global$Run.compileSources(Global.scala:1393)
[breeze] 	at scala.tools.nsc.Global$Run.compile(Global.scala:1487)
[breeze] 	at xsbt.CachedCompiler0.run(CompilerInterface.scala:116)
[breeze] 	at xsbt.CachedCompiler0.run(CompilerInterface.scala:95)
[breeze] 	at xsbt.CompilerInterface.run(CompilerInterface.scala:26)
[breeze] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[breeze] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[breeze] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[breeze] 	at java.lang.reflect.Method.invoke(Method.java:498)
[breeze] 	at sbt.compiler.AnalyzingCompiler.call(AnalyzingCompiler.scala:107)
[breeze] 	at sbt.compiler.AnalyzingCompiler.compile(AnalyzingCompiler.scala:53)
[breeze] 	at sbt.compiler.AnalyzingCompiler.compile(AnalyzingCompiler.scala:47)
[breeze] 	at sbt.compiler.MixedAnalyzingCompiler$$anonfun$compileScala$1$1.apply$mcV$sp(MixedAnalyzingCompiler.scala:50)
[breeze] 	at sbt.compiler.MixedAnalyzingCompiler$$anonfun$compileScala$1$1.apply(MixedAnalyzingCompiler.scala:50)
[breeze] 	at sbt.compiler.MixedAnalyzingCompiler$$anonfun$compileScala$1$1.apply(MixedAnalyzingCompiler.scala:50)
[breeze] 	at sbt.compiler.MixedAnalyzingCompiler.timed(MixedAnalyzingCompiler.scala:74)
[breeze] 	at sbt.compiler.MixedAnalyzingCompiler.compileScala$1(MixedAnalyzingCompiler.scala:49)
[breeze] 	at sbt.compiler.MixedAnalyzingCompiler.compile(MixedAnalyzingCompiler.scala:64)
[breeze] 	at sbt.compiler.IC$$anonfun$compileInternal$1.apply(IncrementalCompiler.scala:160)
[breeze] 	at sbt.compiler.IC$$anonfun$compileInternal$1.apply(IncrementalCompiler.scala:160)
[breeze] 	at sbt.inc.IncrementalCompile$$anonfun$doCompile$1.apply(Compile.scala:66)
[breeze] 	at sbt.inc.IncrementalCompile$$anonfun$doCompile$1.apply(Compile.scala:64)
[breeze] 	at sbt.inc.IncrementalCommon.cycle(IncrementalCommon.scala:32)
[breeze] 	at sbt.inc.Incremental$$anonfun$1.apply(Incremental.scala:72)
[breeze] 	at sbt.inc.Incremental$$anonfun$1.apply(Incremental.scala:71)
[breeze] 	at sbt.inc.Incremental$.manageClassfiles(Incremental.scala:99)
[breeze] 	at sbt.inc.Incremental$.compile(Incremental.scala:71)
[breeze] 	at sbt.inc.IncrementalCompile$.apply(Compile.scala:54)
[breeze] 	at sbt.compiler.IC$.compileInternal(IncrementalCompiler.scala:160)
[breeze] 	at sbt.compiler.IC$.incrementalCompile(IncrementalCompiler.scala:138)
[breeze] 	at sbt.Compiler$.compile(Compiler.scala:155)
[breeze] 	at sbt.Compiler$.compile(Compiler.scala:141)
[breeze] 	at sbt.Defaults$.sbt$Defaults$$compileIncrementalTaskImpl(Defaults.scala:879)
[breeze] 	at sbt.Defaults$$anonfun$compileIncrementalTask$1.apply(Defaults.scala:870)
[breeze] 	at sbt.Defaults$$anonfun$compileIncrementalTask$1.apply(Defaults.scala:868)
[breeze] 	at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
[breeze] 	at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
[breeze] 	at sbt.std.Transform$$anon$4.work(System.scala:63)
[breeze] 	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
[breeze] 	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
[breeze] 	at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
[breeze] 	at sbt.Execute.work(Execute.scala:237)
[breeze] 	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
[breeze] 	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
[breeze] 	at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
[breeze] 	at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
[breeze] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[breeze] 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[breeze] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[breeze] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[breeze] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[breeze] 	at java.lang.Thread.run(Thread.java:745)

[adriaanm]

The match was added in scala/scala@9df6d16

[SethTisue]

looks easy to fix by going to None when the inner match fails, as also happened when the inner match was a sub-match of the outer one, before it was made a separate match in the case body

[SethTisue]

Adriaan's skeptical adding test coverage is worth the bother, but I'd like to at least look at what kind of user code is causing this, perhaps adding a test case would be easy

[SethTisue]

without spending too much time investigating, I'm reasonably confident that the case that's blowing up is when there is a Java source file being processed by scalac, and a ClassDef derived from a Java source file hits the pattern match as it passes through some code in macro paradise.

possible approaches to testing this. in order from unit to integration:

1. figure out what the tree that's coming from the Java source file looks like, then write a unit test that synthesizes such a tree and passes it to the code in question
2. write a test that compiles an actual Java source file, gets the tree from it, and then does something comparable to whatever it is macro paradise is doing that results in that tree ending up at this pattern match
3. consider it macro paradise's job to test, since macro paradise is the only currently known way to trigger the problem in practice and we can't depend on it from the scala/scala repo
4. let the community build catch it

1 seems too fragile to provide a real backstop. as for 3, the bug is so specific that even if we notified Eugene B I doubt he'd have the bandwidth to handle it. 2 is the most plausible, but I think Adriaan convinced me it just isn't worth the effort, given the amount of investigation needed and test scaffolding to be constructed.

so let's call it a win for the community build and call it a day. the community build led us right to the problem, and fortunately the fix is obvious.