Skip to content

Copy the strawman FAQ as an overview page #1318

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Apr 30, 2019
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions _data/overviews.yml
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,11 @@
url: "collections-2.13/creating-collections-from-scratch.html"
- title: Conversions Between Java and Scala Collections
url: "collections-2.13/conversions-between-java-and-scala-collections.html"
- title: Migrating a Project to Scala 2.13's Collections
icon: sitemap
url: "core/collections-migration-213.html"
description: "This page describes the main changes for collection users that migrate to Scala
2.13 and shows how to cross-build projects with Scala 2.11 / 2.12 and 2.13."
- title: The Architecture of Scala 2.13’s Collections
icon: sitemap
url: "core/architecture-of-scala-213-collections.html"
Expand Down
231 changes: 231 additions & 0 deletions _overviews/core/collections-migration-213.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,231 @@
---
layout: singlepage-overview
title: Migrating a Project to Scala 2.13's Collections
permalink: /overviews/core/:title.html
---

This document describes the main changes for collection users that migrate to Scala 2.13 and shows
how to cross-build projects with Scala 2.11 / 2.12 and 2.13.

For an in-depth overview of the Scala 2.13 collections library, see the [collections guide]({{ site.baseurl }}/overviews/collections-2.13/introduction.html). The implementation details of the 2.13 collections are explained in the document [the architecture of Scala collections]({{ site.baseurl }}/overviews/core/architecture-of-scala-213-collections.html).

The most important changes in the Scala 2.13 collections library are:
- Transformation methods no longer have an implicit `CanBuildFrom` parameter. This makes the library easier to understand (in source code, Scaladoc, and IDE code completion). It also makes compiling user code more efficient.
- The type hierarchy is simplified. `Traversable` no longer exists, only `Iterable`.
- The `to[Collection]` method was replaced by the `to(Collection)` method.
- Views have been vastly simplified and work reliably now. They no longer extend their corresponding collection type, for example, an `IndexedSeqView` no longer extends `IndexedSeq`.
- `collection.breakOut` no longer exists, use `.view` and `.to(Collection)` instead.
- Immutable hash sets and hash maps have a new implementation (`ChampHashSet` and `ChampHashMap`, based on the ["CHAMP" encoding](https://michael.steindorfer.name/publications/oopsla15.pdf)).
- New collection types:
- `immutable.ArraySeq` is an effectively immutable sequence that wraps an array
- `immutable.LazyList` is a linked list that is lazy in its state, i.e., whether it's empty or non-empty. This allows creating a `LazyList` without evaluating the `head` element. `immutable.Stream`, which has a strict `head` and a lazy `tail`, is deprecated.
- Deprecated collections were removed (`MutableList`, `immutable.Stack`, others)
- Parallel collections are now in a separate hierarchy in a [separate module](https://github.com/scala/scala-parallel-collections).
- The `scala.jdk.StreamConverters` object provides extension methods to create (sequential or parallel) Java 8 streams for Scala collections.
- `scala.Seq` is now an alias for `scala.collection.immutable.Seq` (no longer `scala.collection.Seq`). Note that this also changes the type of Scala varargs methods.

## Tools for migrating and cross-building

The [scala-collection-compat](https://github.com/scala/scala-collection-compat) is a library released for 2.11, 2.12 and 2.13 that provides some of the new APIs from Scala 2.13 for the older versions. This simplifies cross-building projects.

The module also provides [migratrion rules](https://github.com/scala/scala-collection-compat#migration-tool) for [scalafix](https://scalacenter.github.io/scalafix/docs/users/installation.html) that can update a project's source code to work with the 2.13 collections library.

## What are the breaking changes?

The following table summarizes the breaking changes. The "Automatic Migration Rule" column gives the name of the migration rule that can be used to automatically update old code to the new expected form.

| Description | Old Code | New Code | Automatic Migration Rule |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the table does not render well and it only has few entries, we could just replace table rows with paragraphs:

### Operation `to[C[_]]`

Operation `to[C[_]]` has been removed.

Scala 2.12 code:

```scala
xs.to[List]
```

Scala 2.13 code:

```scala
xs.to(List)
```

Automatic migration rules: `Collection213Upgrade` and `Collections213CrossCompat`

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i left the tables for now. not sure what's the best way to fix it... is there something like a scroll-div we can use? for code blocks that is done automatically.

| ----------- | -------- | -------- | ------------------------ |
| Method `to[C[_]]` has been removed (it might be reintroduced but deprecated, though) | `xs.to[List]` | `xs.to(List)` | `Collection213Upgrade`, `Collections213CrossCompat` |
| `mapValues` and `filterKeys` now return a `MapView` instead of a `Map` | `kvs.mapValues(f)` | `kvs.mapValues(f).toMap` | `RoughlyMapValues` |
| `Iterable` no longer has a `sameElements` operation | `xs1.sameElements(xs2)` | `xs1.iterator.sameElements(xs2)` | `Collection213Upgrade`, `Collections213CrossCompat` |
| `collection.breakOut` no longer exists | `val xs: List[Int] = ys.map(f)(collection.breakOut)` | `val xs = ys.iterator.map(f).to(List)` | `Collection213Upgrade` |
| `zip` on `Map[K, V]` now returns an `Iterable` | `map.zip(iterable)` | `map.zip(iterable).toMap` | `Collection213Experimental` |
| `ArrayBuilder.make` does not accept parens anymore | `ArrayBuilder.make[Int]()` | `ArrayBuilder.make[Int]` | `Collection213Upgrade`, `Collections213CrossCompat` |

Some classes have been removed, made private or have no equivalent in the new design:

- `ArrayStack`
- `mutable.FlatHashTable`
- `mutable.HashTable`
- `History`
- `Immutable`
- `IndexedSeqOptimized`
- `LazyBuilder`
- `mutable.LinearSeq`
- `LinkedEntry`
- `MapBuilder`
- `Mutable`
- `MutableList`
- `Publisher`
- `ResizableArray`
- `RevertibleHistory`
- `SeqForwarder`
- `SetBuilder`
- `Sizing`
- `SliceInterval`
- `StackBuilder`
- `StreamView`
- `Subscriber`
- `Undoable`
- `WrappedArrayBuilder`

Other notable changes are:

- `Iterable.partition` invokes `iterator` twice on non-strict collections and assumes it gets two iterators over the same elements. Strict subclasses override `partition` do perform only a single traversal
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How was it before in Stream?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i don't know :) didn't check.

- `scala.Seq[+A]` is now `scala.collection.immutable.Seq[A]` (this also affects varargs methods).
- Equality between collections is not anymore defined at the level of `Iterable`. It is defined separately in the `Set`, `Seq` and `Map` branches. Another consequence is that `Iterable` does not anymore have a `canEqual` method.
- The new collections makes more use of overloading. You can find more information about the motivation
behind this choice [here](http://scala-lang.org/blog/2017/05/30/tribulations-canbuildfrom.html). For instance, `Map.map` is overloaded:

scala> Map(1 -> "a").map
def map[B](f: ((Int, String)) => B): scala.collection.immutable.Iterable[B]
def map[K2, V2](f: ((Int, String)) => (K2, V2)): scala.collection.immutable.Map[K2,V2]

Type inference has been improved so that `Map(1 -> "a").map(x => (x._1 + 1, x._2))` works, the compiler can infer the parameter type for the function literal. However, using a method reference in 2.13.0-M4 (improvement are on the way for 2.13.0) does not work, and an explicit eta-expansion is necessary:

scala> def f(t: (Int, String)) = (t._1 + 1, t._2)
scala> Map(1 -> "a").map(f)
^
error: missing argument list for method f
Unapplied methods are only converted to functions when a function type is expected.
You can make this conversion explicit by writing `f _` or `f(_)` instead of `f`.
scala> Map(1 -> "a").map(f _)
res10: scala.collection.immutable.Map[Int,String] = ChampHashMap(2 -> a)
- `View`s have been completely redesigned and we expect their usage to have a more predictable evaluation model.
You can read more about the new design [here](http://scala-lang.org/blog/2017/11/28/view-based-collections.html).
- `mutable.ArraySeq` (which wraps an `Array[AnyRef]` in 2.12, meaning that primitives were boxed in the array) can now wrap boxed and unboxed arrays. `mutable.ArraySeq` in 2.13 is in fact equivalent to `WrappedArray` in 2.12, there are specialized subclasses for primitive arrays. Note that a `mutable.ArraySeq` can be used either way for primitive arrays (TODO: document how). `WrappedArray` is deprecated.
- There is no "default" `Factory` (previously known as `[A, C] => CanBuildFrom[Nothing, A, C]`): use `Factory[A, Vector[A]]` explicitly instead.

## Breaking changes with old syntax still supported

The following table lists the changes that continue to work with a deprecation warning.

| Description | Old Code | New Code | Automatic Migration Rule |
| ----------- | -------- | -------- | ------------------------ |
| `collection.Set/Map` no longer have `+` and `-` operations | `xs + 1 - 2` | `xs ++ Set(1) -- Set(2)` | `Collection213Experimental` |
| `collection.Map` no longer have `--` operation | `map -- keys` | `map.to(immutable.Map) -- keys` | |
| `immutable.Set/Map`: the `+` operation no longer has an overload accepting multiple values | `Set(1) + (2, 3)` | `Set(1) + 2 + 3` | `Collection213Upgrade`, `Collections213CrossCompat` |
| `mutable.Map` no longer have an `updated` method | `mutable.Map(1 -> 2).updated(1, 3)` | `mutable.Map(1 -> 2).clone() += 1 -> 3` | `Collection213Upgrade`, `Collections213CrossCompat` |
| `mutable.Set/Map` no longer have a `+` operation | `mutable.Set(1) + 2` | `mutable.Set(1).clone() += 2` | `Collection213Upgrade`, `Collections213CrossCompat` |
| `SortedSet`: the `to`, `until` and `from` methods are now called `rangeTo`, `rangeUntil` and `rangeFrom`, respectively | `xs.until(42)` | `xs.rangeUntil(42)` | |
| `Traversable` and `TraversableOnce` are replaced with `Iterable` and `IterableOnce`, respectively | `def f(xs: Traversable[Int]): Unit` | `def f(xs: Iterable[Int]): Unit` | `Collection213Upgrade`, `Collections213CrossCompat` |
| `Stream` is replaced with `LazyList` | `Stream.from(1)` | `LazyList.from(1)` | `Collection213Roughly` |
| `Seq#union` is replaced with `concat` | `xs.union(ys)` | `xs.concat(ys)` | |
| `Stream#append` is replaced with `lazyAppendAll` | `xs.append(ys)` | `xs.lazyAppendedAll(ys)` | `Collection213Upgrade`, `Collections213CrossCompat` |
| `IterableOnce#toIterator` is replaced with `IterableOnce#iterator` | `xs.toIterator` | `xs.iterator` | `Collection213Upgrade`, `Collections213CrossCompat` |
| `copyToBuffer` has been deprecated | `xs.copyToBuffer(buffer)` | `buffer ++= xs` | `Collection213Upgrade`, `Collections213CrossCompat` |
| `TupleNZipped` has been replaced with `LazyZipN` | `(xs, ys).zipped` | `xs.lazyZip(ys)` | `Collection213Upgrade` |
| `retain` has been renamed to `filterInPlace` | `xs.retain(f)` | `xs.filterInPlace(f.tupled)` | `Collection213Upgrade` |
| `:/` and `/:` operators have been deprecated | `(xs :\ y)(f)` | `xs.foldRight(y)(f)` | `Collection213Upgrade`, `Collections213CrossCompat` |
| `companion` operation has been renamed to `iterableFactory` | `xs.companion` | `xs.iterableFactory` | |

## Deprecated things in 2.12 that have been removed in 2.13

- `collection.convert.JavaConversions`. Use `collection.convert.JavaConverters` instead ;
- `collection.mutable.MutableList` (was not deprecated in 2.12 but was considered to be an implementation detail for implementing other collections). Use an `ArrayDeque` instead, or a `List` and a `var` ;
- `collection.immutable.Stack`. Use a `List` instead ;
- `StackProxy`, `MapProxy`, `SetProxy`, `SeqProxy`, etc. No replacement ;
- `SynchronizedMap`, `SynchronizedBuffer`, etc. Use `java.util.concurrent` instead ;

## Are there new collection types?

`scala.collection.immutable.ArraySeq` is an immutable sequence backed by an array. It is used to pass varargs parameters.

The [`scala-collection-contrib`](https://github.com/scala/scala-collection-contrib) module provides decorators enriching the collections with new operations. You can
think of this artifact as an incubator: if we get evidence that these operations should be part of the core,
we might eventually move them.

The following collections are provided:

- `MultiSet` (both mutable and immutable)
- `SortedMultiSet` (both mutable and immutable)
- `MultiDict` (both mutable and immutable)
- `SortedMultiDict` (both mutable and immutable)

## Are there new operations on collections?

The following new partitioning operations are available:

~~~ scala
def groupMap[K, B](key: A => K)(f: A => B): Map[K, CC[B]] // (Where `CC` can be `List`, for instance)
def groupMapReduce[K, B](key: A => K)(f: A => B)(g: (B, B) => B): Map[K, B]
~~~

`groupMap` is equivalent to `groupBy(key).mapValues(_.map(f))`.

`groupMapReduce` is equivalent to `groupBy(key).mapValues(_.map(f).reduce(g))`.

Mutable collections now have transformation operations that modify the collection in place:

~~~ scala
def mapInPlace(f: A => A): this.type
def flatMapInPlace(f: A => IterableOnce[A]): this.type
def filterInPlace(p: A => Boolean): this.type
def patchInPlace(from: Int, patch: scala.collection.Seq[A], replaced: Int): this.type
~~~

Other new operations are `distinctBy` and `partitionMap`

~~~ scala
def distinctBy[B](f: A => B): C // `C` can be `List[Int]`, for instance
def partitionMap[A1, A2](f: A => Either[A1, A2]): (CC[A1], CC[A2]) // `CC` can be `List`, for instance
~~~

Last, additional operations are provided by the `scala-collection-contrib` module. You can
think of this artifact as an incubator: if we get evidence that these operations should be part of the core,
we might eventually move them.

The new operations are provided via an implicit enrichment. You need to add the following import to make them
available:

~~~ scala
import strawman.collection.decorators._
~~~

The following operations are provided:

- `Seq`
- `intersperse`
- `Map`
- `zipByKey` / `join` / `zipByKeyWith`
- `mergeByKey` / `fullOuterJoin` / `mergeByKeyWith` / `leftOuterJoin` / `rightOuterJoin`

## Are there new implementations of existing collection types (changes in performance characteristics)?

The default `Set` and `Map` are backed by a `ChampHashSet` and a `ChampHashMap`, respectively. The performance characteristics are the same but the
operation implementations are faster. These data structures also have a lower memory footprint.

`mutable.Queue` and `mutable.Stack` now use `mutable.ArrayDeque`. This data structure supports constant time index access, and amortized constant time
insert and remove operations.

## How do I cross-build my project against Scala 2.12 and Scala 2.13?

Most usages of collections are compatible and can cross-compile 2.12 and 2.13 (at the cost of some warnings, sometimes).

If you cannot get your code to cross-compile, there are various solutions:
- You can use the [`scala-collection-compat`](https://github.com/scala/scala-collection-compat) library, which makes some of 2.13's APIs available to 2.11 and 2.12. This solution does not always work, for example if your library implements custom collection types.
- You can maintain a separate branch with the changes for 2.13 and publish releases for 2.13 from this branch.
- You can put source files that don't cross-compile in separate directories and configure sbt to assemble the sources according to the Scala version (see also the examples below):

// Adds a `src/main/scala-2.13+` source directory for Scala 2.13 and newer
// and a `src/main/scala-2.13-` source directory for Scala version older than 2.13
unmanagedSourceDirectories in Compile += {
val sourceDir = (sourceDirectory in Compile).value
CrossVersion.partialVersion(scalaVersion.value) match {
case Some((2, n)) if n >= 13 => sourceDir / "scala-2.13+"
case _ => sourceDir / "scala-2.13-"
}
}

Examples of libraries that cross-compile with separate source directories:
- https://github.com/scala/scala-parser-combinators/pull/152
- https://github.com/scala/scala-xml/pull/222
- Some other examples are listed here: https://github.com/scala/community-builds/issues/710

# Collection Implementers

To learn about differences when implementing custom collection types or operations, see the following documents:
- [The architecture of Scala collections]({{ site.baseurl }}/overviews/core/architecture-of-scala-213-collections.html)
- [Implementing custom collections]({{ site.baseurl }}/overviews/core/custom-collections.html)
- [Adding custom collection operations]({{ site.baseurl }}/overviews/core/custom-collection-operations.html)