Skip to content

Commit 182408e

Browse files
lrytzlrytz
committed
more details on object init locking, lambda outer captures, sam conversions
1 parent 4fd0886 commit 182408e

File tree

1 file changed

+114
-37
lines changed

1 file changed

+114
-37
lines changed

‎hand-written.md

Lines changed: 114 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -78,17 +78,17 @@ Note that the compiler still has quite a bit of magic to perform behind the scen
7878

7979
#### Java 8-style lambdas
8080

81+
-- TODO: break this up into SAM and IndyLambda
82+
8183
Scala 2.12 emits closures in the same style as Java 8, whether they target a `FunctionN` class from the standard library or a user-defined Single Abstract Method (SAM) type. The type checker accepts a function literal as a valid expression for either kind of "function-like" type (built-in or SAM). This improves the experience of using libraries written for Java 8 in Scala.
8284

8385
For example, in the REPL:
8486

85-
```
86-
scala> val runRunnable: Runnable = () => println("Run!")
87-
runRunnable: Runnable = $$Lambda$1073/754978432@7cf283e1
88-
89-
scala> runRunnable.run()
90-
Run!
91-
```
87+
scala> val runRunnable: Runnable = () => println("Run!")
88+
runRunnable: Runnable = $$Lambda$1073/754978432@7cf283e1
89+
90+
scala> runRunnable.run()
91+
Run!
9292

9393
For each lambda the compiler generates a method containing the lambda body, and emits an `invokedynamic` that will spin up a lightweight class for this closure using the JDK's `LambdaMetaFactory`. Note that in the following situations, an anonymous function class is still synthesized at compile-time:
9494

@@ -115,13 +115,11 @@ For now, we recommend using `-Ypartial-unification` over `-Xexperimental`, as th
115115

116116
With [JEP-118](http://openjdk.java.net/jeps/118), parameter names can be stored in class files and be queried at runtime using Java reflection. A quick REPL session shows this in action:
117117

118-
```
119-
scala> case class Person(name: String, age: Int)
120-
defined class Person
121-
122-
scala> val paramNames = classOf[Person].getConstructors.head.getParameters.toList
123-
paramNames: List[java.lang.reflect.Parameter] = List(final java.lang.String name, final int age)
124-
```
118+
scala> case class Person(name: String, age: Int)
119+
defined class Person
120+
121+
scala> val paramNames = classOf[Person].getConstructors.head.getParameters.toList
122+
paramNames: List[java.lang.reflect.Parameter] = List(final java.lang.String name, final int age)
125123

126124
### Tooling improvements
127125

@@ -143,22 +141,18 @@ The following optimizations are available:
143141

144142
For example, the following code
145143

146-
```scala
147-
def f(a: Int, b: Boolean) = (a, b) match {
148-
case (0, true) => -1
149-
case _ if a < 0 => -a
150-
case _ => a
151-
}
152-
```
144+
def f(a: Int, b: Boolean) = (a, b) match {
145+
case (0, true) => -1
146+
case _ if a < 0 => -a
147+
case _ => a
148+
}
153149

154150
produces, when compiled with `-opt:l:method`, the following bytecode (decompiled using [cfr](http://www.benf.org/other/cfr/)):
155151

156-
```java
157-
public int f(int a, boolean b) {
158-
int n = 0 == a && true == b ? -1 : (a < 0 ? - a : a);
159-
return n;
160-
}
161-
```
152+
public int f(int a, boolean b) {
153+
int n = 0 == a && true == b ? -1 : (a < 0 ? - a : a);
154+
return n;
155+
}
162156

163157
The optimizer supports inlining (disabled by default). With `-opt:l:project` code from source files currently being compiled is inlined, while `-opt:l:classpath` enables inlining code from libraries on the compiler's classpath. Other than methods marked [`@inline`](http://www.scala-lang.org/files/archive/api/2.12.0/scala/inline.html), higher-order methods are inlined if the function argument is a lambda, or a parameter of the caller.
164158

@@ -215,20 +209,101 @@ The Scala library is [free](https://github.com/scala/scala/pull/4443) of [refere
215209

216210
## Breaking changes
217211

218-
### Lambdas and locks
219-
Our new lambda encoding lifts the lambda body to a method in the enclosing class. If your code relies on the old behavior, which spins up a new class for each lambda, you may notice this difference because a lambda invocation will now indirect through the instance of the enclosing class (or in case of an object, its module field). When a lock is held on this instance (e.g., during object initialization), deadlocks may occur that did not happen before. One example of this [surprised users of ScalaCheck](https://github.com/rickynils/scalacheck/issues/290) -- now [fixed](https://github.com/rickynils/scalacheck/pull/294).
212+
### Object initialization locks and lambdas
213+
214+
In Scala 2.11, the body of a lambda is in the `apply` method of the anonymous function class generated at compile time. The new lambda encoding in 2.12 lifts the lambda body into a method in the enclosing class. An invocation of the lambda will therefore indirect through the enclosing class, which may cause deadlocks that did not happen before.
215+
216+
For example, the following code
217+
218+
import scala.concurrent._
219+
import scala.concurrent.duration._
220+
import ExecutionContext.Implicits.global
221+
object O { Await.result(Future(1), 5.seconds) }
222+
223+
compiles to (simplified):
224+
225+
public final class O$ {
226+
public static O$ MODULE$;
227+
public static final int $anonfun$new$1() { return 1; }
228+
public static { new O$(); }
229+
private O$() {
230+
MODULE$ = this;
231+
Await.result(Future.apply(LambdaMetaFactory(Function0, $anonfun$new$1)), DurationInt(5).seconds);
232+
}
233+
}
234+
235+
Accessing `O` for the first time initializes the `O$` class and executes the static initializer (which invokes the instance constructor). Class initialization is guarded by an initialization lock ([Chapter 5.5 in the JVM specification](https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-5.html#jvms-5.5)).
236+
237+
The main thread locks class initialization and spawns the Future. The Future, executed on a different thread, attempts to execute the static lambda body method `$anonfun$new$1`, which also requires initialization of the class `O$`. Because initialization is locked by the main thread, the thread running the future will block. In the meantime, the main thread continues to run `Await.result`, which will block until the future completes, causing the deadlock.
238+
239+
One example of this [surprised the authors of ScalaCheck](https://github.com/rickynils/scalacheck/issues/290) -- now [fixed](https://github.com/rickynils/scalacheck/pull/294).
240+
241+
### Lambdas capturing outer instances
242+
243+
Because lambda bodies are emitted as methods in the enclosing class, a lambda can capture the outer instance in cases where this did not happen in 2.11. This can affect serialization.
244+
245+
The Scala compiler analyzes classes and methods to prevent unnecessary outer captures: unused outer parameters are removed from classes ([#4652](https://github.com/scala/scala/pull/4652)), and methods not accessing any instance members are made static ([#5099](https://github.com/scala/scala/pull/5099)). One known limitation is that the analysis is local to a class and does not cover subclasses.
220246

221-
### SAM types
247+
class C {
248+
def f = () => {
249+
class A extends Serializable
250+
class B extends A
251+
serialize(new A)
252+
}
253+
}
222254

223-
Implicit conversion of function types to SAM types won't kick in as often now, since the compiler's own SAM conversion takes priority:
255+
In this example, the classes `A` and `B` are first lifted into `C`. When flattening the classes to the package level, the `A` obtains an outer pointer to capture the `A` instance. Because `A` has a subclass `B`, the class-level analysis of `A` cannot conclude that the outer parameter is unused (it might be used in `B`).
224256

225-
trait MySam { def apply(x: Int): String }
226-
implicit def unused(fun: Int => String): MySam =
227-
new MySam { def apply(x: Int) = fun(x) }
228-
// uses SAM conversion, not the `unused` implicit
229-
val sammy: MySam = (_: Int).toString
257+
Serializing the `A` instance attempts to serialize the outer field, which causes a `NotSerializableException: C`.
258+
259+
260+
### SAM conversion precedes implicits
261+
262+
The [SAM conversion](http://www.scala-lang.org/files/archive/spec/2.12/06-expressions.html#sam-conversion) built into the type system takes priority over implicit conversion of function types to SAM types. This can change the semantics of existing code relying on implicit conversion to SAM types:
263+
264+
trait MySam { def i(): Int }
265+
implicit def convert(fun: () => Int): MySam = new MySam { def i() = 1 }
266+
val sam1: MySam = () => 2 // Uses SAM conversion, not the implicit
267+
sam1.i() // Returns 2
268+
269+
To retain the old behavior, you may compile under `-Xsource:2.11`, use an explicit call to the conversion method, or disqualify the type from being a SAM (e.g. by adding a second abstract method).
270+
271+
Note that SAM conversion only applies to lambda expressions, not to arbitrary expressions with Scala `FunctionN` types:
272+
273+
val fun = () => 2 // Type Function0[Int]
274+
val sam2: MySam = fun // Uses implicit conversion
275+
sam2.i() // Returns 1
276+
277+
278+
### SAM conversion in overloading resolution
279+
280+
In order to improve source compatibility, overloading resolution has been adapted to prefer methods with `Function`-typed arguments over methods with parameters of SAM types. The following example is identical in Scala 2.11 and 2.12:
281+
282+
scala> object T {
283+
| def m(f: () => Unit) = 0
284+
| def m(r: Runnable) = 1
285+
| }
286+
287+
scala> val f = () => ()
288+
289+
scala> T.m(f)
290+
res0: Int = 0
291+
292+
In Scala 2.11, the first alternative is chosen because it is the only applicable method. In Scala 2.12, both methods are applicable, therefore [overloading resolution](http://www.scala-lang.org/files/archive/spec/2.12/06-expressions.html#overloading-resolution) needs to pick the most specific alternative. The specification for [*compatibility*](http://www.scala-lang.org/files/archive/spec/2.12/03-types.html#compatibility) has been updated to consider SAM conversion, so that the first alternative is more specific.
293+
294+
Note that SAM conversion in overloading resolution is always considered, also if the argument expression is not a function literals. This is unlike SAM conversions of expressions themselves, see the previous section. See also the discussion in [scala-dev#158](https://github.com/scala/scala-dev/issues/158).
295+
296+
While the adjustment to overloading resolution improves compatibility, there can be code that compiles in 2.11, but is ambiguous in 2.12:
297+
298+
scala> object T {
299+
| def m(f: () => Unit, o: Object) = 0
300+
| def m(r: Runnable, s: String) = 1
301+
| }
302+
defined object T
303+
304+
scala> T.m(() => (), "")
305+
<console>:13: error: ambiguous reference to overloaded definition
230306

231-
To retain the old behavior, you may compile under `-Xsource:2.11`, or disqualify the type from being a SAM (e.g. by adding a second abstract method).
232307

233308
### Inferred types for `val` (and `lazy val`)
234309

@@ -240,6 +315,8 @@ You can get the old behavior with `-Xsource:2.11`. This may be useful for testin
240315

241316
[Lazy vals and objects](https://github.com/scala/scala/pull/5294) have been reworked, and those defined in methods now use a [more efficient representation](https://github.com/scala/scala/pull/5374) that allows synchronization on the holder of the `lazy val`, instead of the surrounding class (as in Dotty).
242317

318+
-- TODO: why is this in the "inferred types" section, why under "breaking changes"? This should go somewhere else.
319+
243320
### Changed syntax trees (affects macro and compiler plugin authors)
244321

245322
PR [#4794](https://github.com/scala/scala/pull/4749) changed the syntax trees for selections of statically accessible symbols. For example, a selection of `Predef` no longer has the shape `q"scala.this.Predef"` but simply `q"scala.Predef"`. Macros and compiler plugins matching on the old tree shape need to be adjusted.

0 commit comments

Comments
 (0)