Deterministic Compilation / Reproducible Builds

[heartsucker]

This issue is being opened in response to this thread.

I want to have reproducible artifacts output from scalac and sbt so that I can:

1. Have multiple, independent CI servers generate a build, sign it, and require multiple signatures for any artifact to be valid later in a pipeline
2. Have independent sources be able to compile and verify that the output JAR / class files are identical to what they compiled locally

Because this could be complicated by faffing about too much with a build.scala like embedding the current time somewhere, let's assume that not all builds will be reproducible and that doing so might require minimal tweaking.

It would be a stretch to require incremental compilation to be reproducible, so what would be sufficient is that any sbt clean compile outputs the exact same things.

[dragos]

I'm afraid this ticket is not going to be very actionable since it falls between Sbt and Scala.

In my opinion the output of the compiler should be deterministic regardless of full vs incremental builds. Incremental compilation is not even a concept in the compiler: scalac simply takes a set of sources and compiles them, without any knowledge about what constitutes a full project. From the point of view of the compiler only the command line arguments are relevant.

In the current setup, assuming Sbt is sorting source files prior to sending them to scalac, full builds would be deterministic, but I doubt we can close the ticket in confidence.

[jvican]

I'll be commenting in the original Discourse thread about reproducibility in Zinc and some of my insights on how to achieve it. It's something I'm focusing on, and others volunteers are too. So far, there's progress.

[retronym]

I have collected some definitions of "stability" and test cases that show where we do fail to meet the strict definition (same output regardless of source file order or partial compilation of subsets of files). I'm working on fixing those at the moment.