Skip to content

Commit 1e0fdda

Browse files
lrytzlrytz
authored
Merge pull request #1318 from lrytz/collections-migration-213
Copy the strawman FAQ as an overview page
2 parents 1d44b0e + ba9ba4a commit 1e0fdda

File tree

2 files changed

+244
-0
lines changed

2 files changed

+244
-0
lines changed

‎_data/overviews.yml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,11 @@
8282
url: "collections-2.13/creating-collections-from-scratch.html"
8383
- title: Conversions Between Java and Scala Collections
8484
url: "collections-2.13/conversions-between-java-and-scala-collections.html"
85+
- title: Migrating a Project to Scala 2.13's Collections
86+
icon: sitemap
87+
url: "core/collections-migration-213.html"
88+
description: "This page describes the main changes for collection users that migrate to Scala
89+
2.13 and shows how to cross-build projects with Scala 2.11 / 2.12 and 2.13."
8590
- title: The Architecture of Scala 2.13’s Collections
8691
icon: sitemap
8792
url: "core/architecture-of-scala-213-collections.html"

‎_overviews/core/collections-migration-213.md

Lines changed: 239 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,239 @@
1+
---
2+
layout: singlepage-overview
3+
title: Migrating a Project to Scala 2.13's Collections
4+
permalink: /overviews/core/:title.html
5+
---
6+
7+
This document describes the main changes for collection users that migrate to Scala 2.13 and shows
8+
how to cross-build projects with Scala 2.11 / 2.12 and 2.13.
9+
10+
For an in-depth overview of the Scala 2.13 collections library, see the [collections guide]({{ site.baseurl }}/overviews/collections-2.13/introduction.html). The implementation details of the 2.13 collections are explained in the document [the architecture of Scala collections]({{ site.baseurl }}/overviews/core/architecture-of-scala-213-collections.html).
11+
12+
The most important changes in the Scala 2.13 collections library are:
13+
- Transformation methods no longer have an implicit `CanBuildFrom` parameter. This makes the library easier to understand (in source code, Scaladoc, and IDE code completion). It also makes compiling user code more efficient.
14+
- The type hierarchy is simplified. `Traversable` no longer exists, only `Iterable`.
15+
- The `to[Collection]` method was replaced by the `to(Collection)` method.
16+
- Views have been vastly simplified and work reliably now. They no longer extend their corresponding collection type, for example, an `IndexedSeqView` no longer extends `IndexedSeq`.
17+
- `collection.breakOut` no longer exists, use `.view` and `.to(Collection)` instead.
18+
- Immutable hash sets and hash maps have a new implementation (`ChampHashSet` and `ChampHashMap`, based on the ["CHAMP" encoding](https://michael.steindorfer.name/publications/oopsla15.pdf)).
19+
- New collection types:
20+
- `immutable.ArraySeq` is an effectively immutable sequence that wraps an array
21+
- `immutable.LazyList` is a linked list that is lazy in its state, i.e., whether it's empty or non-empty. This allows creating a `LazyList` without evaluating the `head` element. `immutable.Stream`, which has a strict `head` and a lazy `tail`, is deprecated.
22+
- Deprecated collections were removed (`MutableList`, `immutable.Stack`, others)
23+
- Parallel collections are now in a separate hierarchy in a [separate module](https://github.com/scala/scala-parallel-collections).
24+
- The `scala.jdk.StreamConverters` object provides extension methods to create (sequential or parallel) Java 8 streams for Scala collections.
25+
- `scala.Seq` is now an alias for `scala.collection.immutable.Seq` (no longer `scala.collection.Seq`). Note that this also changes the type of Scala varargs methods.
26+
27+
## Tools for migrating and cross-building
28+
29+
The [scala-collection-compat](https://github.com/scala/scala-collection-compat) is a library released for 2.11, 2.12 and 2.13 that provides some of the new APIs from Scala 2.13 for the older versions. This simplifies cross-building projects.
30+
31+
The module also provides [migratrion rules](https://github.com/scala/scala-collection-compat#migration-tool) for [scalafix](https://scalacenter.github.io/scalafix/docs/users/installation.html) that can update a project's source code to work with the 2.13 collections library.
32+
33+
## What are the breaking changes?
34+
35+
The following table summarizes the breaking changes. The "Automatic Migration Rule" column gives the name of the migration rule that can be used to automatically update old code to the new expected form.
36+
37+
<div style="overflow:auto;" markdown="block">
38+
39+
| Description | Old Code | New Code | Automatic Migration Rule |
40+
| ----------- | -------- | -------- | ------------------------ |
41+
| Method `to[C[_]]` has been removed (it might be reintroduced but deprecated, though) | `xs.to[List]` | `xs.to(List)` | `Collection213Upgrade`, `Collections213CrossCompat` |
42+
| `mapValues` and `filterKeys` now return a `MapView` instead of a `Map` | `kvs.mapValues(f)` | `kvs.mapValues(f).toMap` | `RoughlyMapValues` |
43+
| `Iterable` no longer has a `sameElements` operation | `xs1.sameElements(xs2)` | `xs1.iterator.sameElements(xs2)` | `Collection213Upgrade`, `Collections213CrossCompat` |
44+
| `collection.breakOut` no longer exists | `val xs: List[Int] = ys.map(f)(collection.breakOut)` | `val xs = ys.iterator.map(f).to(List)` | `Collection213Upgrade` |
45+
| `zip` on `Map[K, V]` now returns an `Iterable` | `map.zip(iterable)` | `map.zip(iterable).toMap` | `Collection213Experimental` |
46+
| `ArrayBuilder.make` does not accept parens anymore | `ArrayBuilder.make[Int]()` | `ArrayBuilder.make[Int]` | `Collection213Upgrade`, `Collections213CrossCompat` |
47+
48+
</div>
49+
50+
Some classes have been removed, made private or have no equivalent in the new design:
51+
52+
- `ArrayStack`
53+
- `mutable.FlatHashTable`
54+
- `mutable.HashTable`
55+
- `History`
56+
- `Immutable`
57+
- `IndexedSeqOptimized`
58+
- `LazyBuilder`
59+
- `mutable.LinearSeq`
60+
- `LinkedEntry`
61+
- `MapBuilder`
62+
- `Mutable`
63+
- `MutableList`
64+
- `Publisher`
65+
- `ResizableArray`
66+
- `RevertibleHistory`
67+
- `SeqForwarder`
68+
- `SetBuilder`
69+
- `Sizing`
70+
- `SliceInterval`
71+
- `StackBuilder`
72+
- `StreamView`
73+
- `Subscriber`
74+
- `Undoable`
75+
- `WrappedArrayBuilder`
76+
77+
Other notable changes are:
78+
79+
- `Iterable.partition` invokes `iterator` twice on non-strict collections and assumes it gets two iterators over the same elements. Strict subclasses override `partition` do perform only a single traversal
80+
- `scala.Seq[+A]` is now `scala.collection.immutable.Seq[A]` (this also affects varargs methods).
81+
- Equality between collections is not anymore defined at the level of `Iterable`. It is defined separately in the `Set`, `Seq` and `Map` branches. Another consequence is that `Iterable` does not anymore have a `canEqual` method.
82+
- The new collections makes more use of overloading. You can find more information about the motivation
83+
behind this choice [here](http://scala-lang.org/blog/2017/05/30/tribulations-canbuildfrom.html). For instance, `Map.map` is overloaded:
84+
85+
scala> Map(1 -> "a").map
86+
def map[B](f: ((Int, String)) => B): scala.collection.immutable.Iterable[B]
87+
def map[K2, V2](f: ((Int, String)) => (K2, V2)): scala.collection.immutable.Map[K2,V2]
88+
89+
Type inference has been improved so that `Map(1 -> "a").map(x => (x._1 + 1, x._2))` works, the compiler can infer the parameter type for the function literal. However, using a method reference in 2.13.0-M4 (improvement are on the way for 2.13.0) does not work, and an explicit eta-expansion is necessary:
90+
91+
scala> def f(t: (Int, String)) = (t._1 + 1, t._2)
92+
scala> Map(1 -> "a").map(f)
93+
^
94+
error: missing argument list for method f
95+
Unapplied methods are only converted to functions when a function type is expected.
96+
You can make this conversion explicit by writing `f _` or `f(_)` instead of `f`.
97+
scala> Map(1 -> "a").map(f _)
98+
res10: scala.collection.immutable.Map[Int,String] = ChampHashMap(2 -> a)
99+
- `View`s have been completely redesigned and we expect their usage to have a more predictable evaluation model.
100+
You can read more about the new design [here](http://scala-lang.org/blog/2017/11/28/view-based-collections.html).
101+
- `mutable.ArraySeq` (which wraps an `Array[AnyRef]` in 2.12, meaning that primitives were boxed in the array) can now wrap boxed and unboxed arrays. `mutable.ArraySeq` in 2.13 is in fact equivalent to `WrappedArray` in 2.12, there are specialized subclasses for primitive arrays. Note that a `mutable.ArraySeq` can be used either way for primitive arrays (TODO: document how). `WrappedArray` is deprecated.
102+
- There is no "default" `Factory` (previously known as `[A, C] => CanBuildFrom[Nothing, A, C]`): use `Factory[A, Vector[A]]` explicitly instead.
103+
104+
## Breaking changes with old syntax still supported
105+
106+
The following table lists the changes that continue to work with a deprecation warning.
107+
108+
<div style="overflow:auto;" markdown="block">
109+
110+
| Description | Old Code | New Code | Automatic Migration Rule |
111+
| ----------- | -------- | -------- | ------------------------ |
112+
| `collection.Set/Map` no longer have `+` and `-` operations | `xs + 1 - 2` | `xs ++ Set(1) -- Set(2)` | `Collection213Experimental` |
113+
| `collection.Map` no longer have `--` operation | `map -- keys` | `map.to(immutable.Map) -- keys` | |
114+
| `immutable.Set/Map`: the `+` operation no longer has an overload accepting multiple values | `Set(1) + (2, 3)` | `Set(1) + 2 + 3` | `Collection213Upgrade`, `Collections213CrossCompat` |
115+
| `mutable.Map` no longer have an `updated` method | `mutable.Map(1 -> 2).updated(1, 3)` | `mutable.Map(1 -> 2).clone() += 1 -> 3` | `Collection213Upgrade`, `Collections213CrossCompat` |
116+
| `mutable.Set/Map` no longer have a `+` operation | `mutable.Set(1) + 2` | `mutable.Set(1).clone() += 2` | `Collection213Upgrade`, `Collections213CrossCompat` |
117+
| `SortedSet`: the `to`, `until` and `from` methods are now called `rangeTo`, `rangeUntil` and `rangeFrom`, respectively | `xs.until(42)` | `xs.rangeUntil(42)` | |
118+
| `Traversable` and `TraversableOnce` are replaced with `Iterable` and `IterableOnce`, respectively | `def f(xs: Traversable[Int]): Unit` | `def f(xs: Iterable[Int]): Unit` | `Collection213Upgrade`, `Collections213CrossCompat` |
119+
| `Stream` is replaced with `LazyList` | `Stream.from(1)` | `LazyList.from(1)` | `Collection213Roughly` |
120+
| `Seq#union` is replaced with `concat` | `xs.union(ys)` | `xs.concat(ys)` | |
121+
| `Stream#append` is replaced with `lazyAppendAll` | `xs.append(ys)` | `xs.lazyAppendedAll(ys)` | `Collection213Upgrade`, `Collections213CrossCompat` |
122+
| `IterableOnce#toIterator` is replaced with `IterableOnce#iterator` | `xs.toIterator` | `xs.iterator` | `Collection213Upgrade`, `Collections213CrossCompat` |
123+
| `copyToBuffer` has been deprecated | `xs.copyToBuffer(buffer)` | `buffer ++= xs` | `Collection213Upgrade`, `Collections213CrossCompat` |
124+
| `TupleNZipped` has been replaced with `LazyZipN` | `(xs, ys).zipped` | `xs.lazyZip(ys)` | `Collection213Upgrade` |
125+
| `retain` has been renamed to `filterInPlace` | `xs.retain(f)` | `xs.filterInPlace(f.tupled)` | `Collection213Upgrade` |
126+
| `:/` and `/:` operators have been deprecated | `(xs :\ y)(f)` | `xs.foldRight(y)(f)` | `Collection213Upgrade`, `Collections213CrossCompat` |
127+
| `companion` operation has been renamed to `iterableFactory` | `xs.companion` | `xs.iterableFactory` | |
128+
129+
</div>
130+
131+
## Deprecated things in 2.12 that have been removed in 2.13
132+
133+
- `collection.convert.JavaConversions`. Use `collection.convert.JavaConverters` instead ;
134+
- `collection.mutable.MutableList` (was not deprecated in 2.12 but was considered to be an implementation detail for implementing other collections). Use an `ArrayDeque` instead, or a `List` and a `var` ;
135+
- `collection.immutable.Stack`. Use a `List` instead ;
136+
- `StackProxy`, `MapProxy`, `SetProxy`, `SeqProxy`, etc. No replacement ;
137+
- `SynchronizedMap`, `SynchronizedBuffer`, etc. Use `java.util.concurrent` instead ;
138+
139+
## Are there new collection types?
140+
141+
`scala.collection.immutable.ArraySeq` is an immutable sequence backed by an array. It is used to pass varargs parameters.
142+
143+
The [`scala-collection-contrib`](https://github.com/scala/scala-collection-contrib) module provides decorators enriching the collections with new operations. You can
144+
think of this artifact as an incubator: if we get evidence that these operations should be part of the core,
145+
we might eventually move them.
146+
147+
The following collections are provided:
148+
149+
- `MultiSet` (both mutable and immutable)
150+
- `SortedMultiSet` (both mutable and immutable)
151+
- `MultiDict` (both mutable and immutable)
152+
- `SortedMultiDict` (both mutable and immutable)
153+
154+
## Are there new operations on collections?
155+
156+
The following new partitioning operations are available:
157+
158+
~~~ scala
159+
def groupMap[K, B](key: A => K)(f: A => B): Map[K, CC[B]] // (Where `CC` can be `List`, for instance)
160+
def groupMapReduce[K, B](key: A => K)(f: A => B)(g: (B, B) => B): Map[K, B]
161+
~~~
162+
163+
`groupMap` is equivalent to `groupBy(key).mapValues(_.map(f))`.
164+
165+
`groupMapReduce` is equivalent to `groupBy(key).mapValues(_.map(f).reduce(g))`.
166+
167+
Mutable collections now have transformation operations that modify the collection in place:
168+
169+
~~~ scala
170+
def mapInPlace(f: A => A): this.type
171+
def flatMapInPlace(f: A => IterableOnce[A]): this.type
172+
def filterInPlace(p: A => Boolean): this.type
173+
def patchInPlace(from: Int, patch: scala.collection.Seq[A], replaced: Int): this.type
174+
~~~
175+
176+
Other new operations are `distinctBy` and `partitionMap`
177+
178+
~~~ scala
179+
def distinctBy[B](f: A => B): C // `C` can be `List[Int]`, for instance
180+
def partitionMap[A1, A2](f: A => Either[A1, A2]): (CC[A1], CC[A2]) // `CC` can be `List`, for instance
181+
~~~
182+
183+
Last, additional operations are provided by the `scala-collection-contrib` module. You can
184+
think of this artifact as an incubator: if we get evidence that these operations should be part of the core,
185+
we might eventually move them.
186+
187+
The new operations are provided via an implicit enrichment. You need to add the following import to make them
188+
available:
189+
190+
~~~ scala
191+
import strawman.collection.decorators._
192+
~~~
193+
194+
The following operations are provided:
195+
196+
- `Seq`
197+
- `intersperse`
198+
- `Map`
199+
- `zipByKey` / `join` / `zipByKeyWith`
200+
- `mergeByKey` / `fullOuterJoin` / `mergeByKeyWith` / `leftOuterJoin` / `rightOuterJoin`
201+
202+
## Are there new implementations of existing collection types (changes in performance characteristics)?
203+
204+
The default `Set` and `Map` are backed by a `ChampHashSet` and a `ChampHashMap`, respectively. The performance characteristics are the same but the
205+
operation implementations are faster. These data structures also have a lower memory footprint.
206+
207+
`mutable.Queue` and `mutable.Stack` now use `mutable.ArrayDeque`. This data structure supports constant time index access, and amortized constant time
208+
insert and remove operations.
209+
210+
## How do I cross-build my project against Scala 2.12 and Scala 2.13?
211+
212+
Most usages of collections are compatible and can cross-compile 2.12 and 2.13 (at the cost of some warnings, sometimes).
213+
214+
If you cannot get your code to cross-compile, there are various solutions:
215+
- You can use the [`scala-collection-compat`](https://github.com/scala/scala-collection-compat) library, which makes some of 2.13's APIs available to 2.11 and 2.12. This solution does not always work, for example if your library implements custom collection types.
216+
- You can maintain a separate branch with the changes for 2.13 and publish releases for 2.13 from this branch.
217+
- You can put source files that don't cross-compile in separate directories and configure sbt to assemble the sources according to the Scala version (see also the examples below):
218+
219+
// Adds a `src/main/scala-2.13+` source directory for Scala 2.13 and newer
220+
// and a `src/main/scala-2.13-` source directory for Scala version older than 2.13
221+
unmanagedSourceDirectories in Compile += {
222+
val sourceDir = (sourceDirectory in Compile).value
223+
CrossVersion.partialVersion(scalaVersion.value) match {
224+
case Some((2, n)) if n >= 13 => sourceDir / "scala-2.13+"
225+
case _ => sourceDir / "scala-2.13-"
226+
}
227+
}
228+
229+
Examples of libraries that cross-compile with separate source directories:
230+
- https://github.com/scala/scala-parser-combinators/pull/152
231+
- https://github.com/scala/scala-xml/pull/222
232+
- Some other examples are listed here: https://github.com/scala/community-builds/issues/710
233+
234+
# Collection Implementers
235+
236+
To learn about differences when implementing custom collection types or operations, see the following documents:
237+
- [The architecture of Scala collections]({{ site.baseurl }}/overviews/core/architecture-of-scala-213-collections.html)
238+
- [Implementing custom collections]({{ site.baseurl }}/overviews/core/custom-collections.html)
239+
- [Adding custom collection operations]({{ site.baseurl }}/overviews/core/custom-collection-operations.html)

0 commit comments

Comments
 (0)