-Yscala-release support: checking if classes come from stdlib based on jar name; better documentation - v2

Author: prolativ

community-build/src/scala/dotty/communitybuild/projects.scala

@@ -601,7 +601,7 @@ object projects:
     dependencies = () => List(cats, disciplineMunit)
   )
 
-  lazy val catsMtlForwardCompat = catsMtl.forwardCompat.withScalaRelease("3.0")
+  lazy val catsMtlForwardCompat = catsMtl.forwardCompat.copy(compilerVersion = "3.0.2")
 
   lazy val coop = SbtCommunityProject(
     project = "coop",

compiler/src/dotty/tools/dotc/core/classfile/ClassfileParser.scala

@@ -972,7 +972,7 @@ class ClassfileParser(
               def isStdlibClass(cls: ClassDenotation): Boolean =
                 ctx.platform.classPath.findClassFile(cls.fullName.mangledString) match {
                   case Some(entry: ZipArchive#Entry) =>
-                    entry.underlyingSource.map(_.name.startsWith("scala3-library_3-")).getOrElse(false)
+                    entry.underlyingSource.map(_.name.startsWith("scala3-library_")).getOrElse(false)
                   case _ => false
                 }
               val isTastyCompatible = fileTastyVersion.isCompatibleWith(ctx.tastyVersion) || isStdlibClass(classRoot)

docs/docs/reference/language-versions/binary-compatibility.md

@@ -3,12 +3,18 @@ layout: doc-page
 title: "Binary Compatibility"
 ---
 
-Thanks to TASTy files, which are produced during compilation of Scala 3 sources, unlike Scala 2, Scala 3 is backward binary compatible between different minor versions (i.e. binary artifacts produced with Scala 3.x can be consumed by Scala 3.y programs as long as x <= y).
-There are however already some ongoing attempts to make Scala 3 forward binary compatible which means that, with some restrictions, it might be possible to compile Scala code with a newer version of the compiler and then use the produced binaries as dependencies for a project using an older compiler.
+In Scala 2 different minor versions of the compiler were free to change the way how they encode different language features in JVM bytecode so each bump of the compiler's minor version resulted in breaking binary compatibility and if a project had any Scala dependencies they all needed to be (cross-)compiled to the same minor Scala version that was used in that project itself.
+While in Scala 3 the JVM encoding might still change between minor versions, an additional intermediate format of code representation called TASTy (from `Typed Abstract Syntax Tree`) was introduced.
 
-Scala 3.1.2-RC1 adds an experimental `-Yscala-release <release-version>` compiler flag which makes the compiler produce TASTy files that should be possible to use by all Scala 3 compilers in version `<release-version>` or newer (this flag was inspired by how `-release` works for specifying the target version of JDK). More specifically this flag enforces emitting TASTy files in an older format ensuring that:
+TASTy files are produced from Scala 3 sources during compilation, together with classfiles, and they're normally also included in published artifacts. A Scala compiler in version `3.x1.y1` is able to read TASTy files produced by another compiler in version `3.x2.y2` if `x1 >= x2` (assuming two stable versions of the compiler are considered - `SNAPSHOT` or `NIGHTLY` compiler versions can read TASTy in an older stable format but their TASTY versions are not compatible between each other even if the compilers have the same minor version; also compilers in stable versions cannot read TASTy generated by an unstable version). While having TASTy files in an understandable format on its classpath a Scala 3 compiler can generate bytecode for a project's dependencies on the fly.
+
+Being able to bump the compiler version in a project without having to wait for all of its dependencies to do the same is already a big leap forward when compared to Scala 2. However, me might still try to do better, especially from the perspective of authors of libraries.
+If you maintain a library and you would like it to be usable as a dependency for all Scala 3 projects, you would have to always emit TASTy in a version that would be readble by everyone, which would normally mean getting stuck at 3.0.x forever.
+
+To solve this problem a new experimental compiler flag `-Yscala-release <release-version>` (available since 3.1.2-RC1) has been added. Setting this flag makes the compiler produce TASTy files that should be possible to use by all Scala 3 compilers in version `<release-version>` or newer (this flag was inspired by how `-release` works for specifying the target version of JDK). More specifically this enforces emitting TASTy files in an older format ensuring that:
 * the code contains no references to parts of the standard library which were added to the API after `<release-version>` and would crash at runtime when a program is executed with the older version of the standard library on the classpath
-* no dependency found on the classpath during compilation (except for the standard library itself) contains TASTy files produced by a compiler newer than `<release-version>` (otherwise they could potentially leak such disallowed references to the standard library)
+* no dependency found on the classpath during compilation (except for the standard library itself) contains TASTy files produced by a compiler newer than `<release-version>` (otherwise they could potentially leak such disallowed references to the standard library).
+
 If any of the checks above is not fulfilled or for any other reason older TASTy cannot be emitted (e.g. the code uses some new language features which cannot be expressed the the older format) the entire compilation fails (with errors reported for each of such issues).
 
 As this feature is experimental it does not have any special support in build tools yet (at least not in sbt 1.6.1 or lower).
@@ -22,4 +28,6 @@ dependencyOverrides ++= Seq(
   scalaOrganization.value %% "scala3-library" % scalaVersion.value,
   scalaOrganization.value %% "scala3-library_sjs1" % scalaVersion.value // for Scala.js projects
 )
-```
+```
+
+The behaviour of `-Yscala-release` flag might still change in the future, especially it's not guaranteed that every new version of the compiler would be able to generate TASTy in all older formats going back to the one produced by `3.0.x` compiler.