Errors when producing jar using sbt-assembly

[mads-hartmann]

I wanted to give dotty a spin. For my specific scenario I wanted to produce a "fat" jar using sbt-assembly. Unfortunately I got the following two errors:

[info] Including: scala-library-0.1.2-RC1.jar
[info] Including: dotty-library_0.1-0.1.2-RC1.jar
[info] Including: scala-library-2.11.11.jar
[debug] Calculate mappings...
[info] Checking every *.class/*.jar file's SHA-1.
[info] Merging files...
[warn] Merging 'META-INF/MANIFEST.MF' with strategy 'discard'
[debug] Merging 'scala/runtime/ScalaRunTime$.class' with strategy 'deduplicate'
[debug] Merging 'scala/runtime/ScalaRunTime.class' with strategy 'deduplicate'
[error] 2 errors were encountered during merge
java.lang.RuntimeException: deduplicate: different file contents found in the following:
/Users/hartmann/.ivy2/cache/ch.epfl.lamp/dotty-library_0.1/jars/dotty-library_0.1-0.1.2-RC1.jar:scala/runtime/ScalaRunTime$.class
/Users/hartmann/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.11.11.jar:scala/runtime/ScalaRunTime$.class
deduplicate: different file contents found in the following:
/Users/hartmann/.ivy2/cache/ch.epfl.lamp/dotty-library_0.1/jars/dotty-library_0.1-0.1.2-RC1.jar:scala/runtime/ScalaRunTime.class
/Users/hartmann/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.11.11.jar:scala/runtime/ScalaRunTime.class
	at sbtassembly.Assembly$.applyStrategies(Assembly.scala:140)
	at sbtassembly.Assembly$.x$1$lzycompute$1(Assembly.scala:25)
	at sbtassembly.Assembly$.x$1$1(Assembly.scala:23)
	at sbtassembly.Assembly$.stratMapping$lzycompute$1(Assembly.scala:23)
	at sbtassembly.Assembly$.stratMapping$1(Assembly.scala:23)
	at sbtassembly.Assembly$.inputs$lzycompute$1(Assembly.scala:67)
	at sbtassembly.Assembly$.inputs$1(Assembly.scala:57)
	at sbtassembly.Assembly$.apply(Assembly.scala:83)
	at sbtassembly.Assembly$$anonfun$assemblyTask$1.apply(Assembly.scala:240)
	at sbtassembly.Assembly$$anonfun$assemblyTask$1.apply(Assembly.scala:237)
	at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
	at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
	at sbt.std.Transform$$anon$4.work(System.scala:63)
	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
	at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
	at sbt.Execute.work(Execute.scala:237)
	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
	at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
	at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
[error] (*:assembly) deduplicate: different file contents found in the following:
[error] /Users/hartmann/.ivy2/cache/ch.epfl.lamp/dotty-library_0.1/jars/dotty-library_0.1-0.1.2-RC1.jar:scala/runtime/ScalaRunTime$.class
[error] /Users/hartmann/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.11.11.jar:scala/runtime/ScalaRunTime$.class
[error] deduplicate: different file contents found in the following:
[error] /Users/hartmann/.ivy2/cache/ch.epfl.lamp/dotty-library_0.1/jars/dotty-library_0.1-0.1.2-RC1.jar:scala/runtime/ScalaRunTime.class
[error] /Users/hartmann/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.11.11.jar:scala/runtime/ScalaRunTime.class

I've attached a tiny reproducible example. It's the result of creating a new dotty project using sbt new lampepfl/dotty.g8 and then adding sbt-assembly to ./project/plugins.sbt.

I'm happy to help but wanted to report the issue first. I chatted with @felixmulder who encouraged me to create an issue.

Cheers,
Mads

[mads-hartmann]

I was able to fix this by applying a merge strategy in build.sbt like so:

assemblyMergeStrategy in assembly := {
  case PathList(ps @ _*) if ps.last contains "ScalaRunTime" => MergeStrategy.first
  case x =>
    val oldStrategy = (assemblyMergeStrategy in assembly).value
    oldStrategy(x)
}

Seems a bit hacky though ;)

[felixmulder]

@smarter - WDYT? Any ideas?

I'll have a look at this next week otherwise.

[smarter]

dotty-library contains a custom ScalaRuntime, there are several issues with it:

• The only difference with the existing ScalaRuntime is that this one has some support for unboxed arrays of value classes. But the implementation of unboxed arrays of value classes was never finished, and no one is working on it or planning to work on it currently.
• This ScalaRuntime is actually never used when we test dotty, instead we always use the ScalaRuntime from scala-library because scala-library is on the bootclasspath.

I propose simply dropping the custom ScalaRuntime, if someone is interested in working on unboxed arrays of value classes, they could reintroduce it in the future. /cc @DarkDimius

[smarter]

The custom ScalaRuntime got removed in #2827 so this should be fixed now, feel free to reopen if there's still an issue with sbt-assembly.