Release 2.13.0-RC3

[adriaanm]

For every Scala release, make a copy of this file named after the release, and expand the variables.
Ideally this should become a script you can run on your local machine. The only missing piece is programmatic triggering of travis jobs with custom config.

Variables to be expanded in this template:

• SCALA_VER_BASE="2.13.0"
• SCALA_VER_SUFFIX="-RC3"
• SCALA_SHA=a73bacc2499170688eda6e0b2a912be5524da737
• DIST_SHA=4507b42f85a51cec140d1fb230a2ea34d02eb697
• SCALA_VER="$SCALA_VER_BASE$SCALA_VER_SUFFIX"

Key links:

• scala/scala milestone: https://github.com/scala/scala/milestone/80
• scala/bug milestone: https://github.com/scala/bug/milestone/101
• scala/scala-dev milestone: N/A past RC1
• community build: https://scala-ci.typesafe.com/job/scala-2.13.x-integrate-community-build/2218/console

N weeks before the release

• Wind down PR queue. There has to be enough time after the last (non-trivial) PR is merged and the next phase. The core of the eco-system needs time to prepare for the final!
• Triage scala/bug and scala/scala-dev tickets
• Create next scala/scala milestone, move the magical "Merge to 2.13.x" description to it (so Scabot uses it as default for new PRs)
• Close the scala/bug milestone, create next milestone, move pending issues
• Close the scala/scala-dev milestone, create next milestone, move pending issues
• Check PRs assigned to the milestone, also check WIP
• Announce expected release date and current nightly "release candidate" (nightly sha-mangled version) at https://scala-ci.typesafe.com/artifactory/scala-integration/ on https://contributors.scala-lang.org/c/announcements

Release announcement / notes

• Notify community on https://contributors.scala-lang.org/c/announcements
• Review merged PRs, make sure release-notes label is applied appropriately
• PRs with release-notes label must have excellent title & description (title will be pasted literally in release note bullet list)
• Draft release notes (to be published once GitHub tag is pushed)
  • note that GitHub release notes drafts can only be viewed by committers, so use a gist to draft the notes, so the gist can be shared with the community
• On contributors thread, link to release note gist and request feedback
• Prepare PR to https://github.com/scala/scala-lang/ (using scala/make-release-notes which requires a staged release and a pushed tag)
  • documentation/api.md
  • download/index.md
  • _config.yml (update devscalaversion or scalaversion)
  • https://github.com/scala/scala-lang/pull/
• Prepare PR to https://github.com/scala/docs.scala-lang/
  • api/all.md
  • https://github.com/scala/docs.scala-lang/pull/

N days before release

• Announce no more PRs will be merged unless last-minute regressions are found. Re-iterate current nightly sha version for testing.
• Community build
  • won't be anywhere near all green, but we should make sure some nontrivial number of projects are still green
  • run with expected-greens only: https://scala-ci.typesafe.com/job/scala-2.13.x-integrate-community-build/?
  • also do a full run to see if there are any unexpected greens
• Run scala-collections-laws and evaluate results
  • Rex Kerr (@Ichoran) is the keeper and expert on this
• Windows Jenkins job
  • https://scala-ci.typesafe.com/job/scala-2.13.x-integrate-windows/?
• Check any merged PRs accidentally assigned to the next milestone in this branch, and re-assign them to this milestone
• Merge in any older release branch (decided not to)
• Check module versioning (is everything in versions.properties up to date?)
• On major release, bump PickleFormat version

Point of no return

• Once sufficient time has passed since last merged PR (1-2 weeks depending on whether it's a maintenance or development branch), and core projects have tried out the candidate nightly, it's time to cut the release!
• Make sure there are no stray staging repos on sonatype
• Trigger a build on travis, specify SCALA_VER_BASE and SCALA_VER_SUFFIX in the custom config: https://travis-ci.org/scala/scala/jobs/
  • config: before_install: export SCALA_VER_BASE=$SCALA_VER_BASE SCALA_VER_SUFFIX=$SCALA_VER_SUFFIX)
  • Check the build status on https://github.com/scala/scala/commits/2.13.x
  • Check that the scala/scala job also triggered a following scala/scala-dist job: https://travis-ci.org/scala/scala/builds/538742305
  • Note the scala/scala commit (the full 40-char sha!)
  • Create the scala/scala tag locally: git tag -s -m "Scala $SCALA_VER" v$SCALA_VER $SCALA_SHA
  • Note the scala/scala-dist commit (the full 40-char sha!)
  • Create scala-dist tag locally: git tag -s -m "Scala $SCALA_VER" v$SCALA_VER $DIST_SHA
  • Note the repos to be promoted after tag is cut (see travis log)
    • https://oss.sonatype.org/content/repositories/orgscala-lang-1763
• Sanity check jar/pom
  • https://oss.sonatype.org/content/repositories/staging/org/scala-lang/scala-compiler/$SCALA_VER/
  • in particular, if the release was staged multiple times, double check that https://oss.sonatype.org/content/repositories/staging/ has the files from the most recent build
• Trigger scala-dist jobs on travis (https://travis-ci.org/scala/scala-dist) with custom config. must use full-length SHAs! https://travis-ci.org/scala/scala-dist/builds/
  • before_install: export version=$SCALA_VER scala_sha=$SCALA_SHA mode=release
• Remember, tags are forever, so are maven artifacts (even staged ones, as they could end up in local caches) and S3 uploads (S3 buckets can be changed, but it can takes days to become consistent)
• Push scala/scala tag: git push https://github.com/scala/scala.git v$SCALA_VER
• Add release notes to https://github.com/scala/scala/releases/tag/v$SCALA_VER
• Push scala/scala-dist tag: git push https://github.com/scala/scala-dist.git v$SCALA_VER
• Trigger two scala-dist jobs on travis (https://travis-ci.org/scala/scala-dist) with custom config. must use full-length SHAs!
  • before_install: export version=$SCALA_VER scala_sha=$SCALA_SHA mode=archives: https://travis-ci.org/scala/scala-dist/builds/
  • before_install: export version=$SCALA_VER scala_sha=$SCALA_SHA mode=update-api: https://travis-ci.org/scala/scala-dist/builds/
• Promote staging repos: st_stagingRepoPromote orgscala-lang-NNN

Check availability

• Check release on sonatype: https://oss.sonatype.org/content/repositories/releases/org/scala-lang/scala-compiler/$SCALA_VER/
• Check the release on maven central: http://central.maven.org/maven2/org/scala-lang/scala-compiler/$SCALA_VER/
• Check the release on maven search (takes longer): http://search.maven.org/#search%7Cga%7C1%7Cg%3Aorg.scala-lang%20v%3A$SCALA_VER

When everything is on maven central

• make sure the draft release notes are on GitHub tag: https://github.com/scala/scala/releases/tag/v$SCALA_VER
• Pre-announce the release on https://contributors.scala-lang.org/c/announcements
• On 2.13.0 only: (manually) update the current symlink for the API docs
  • https://github.com/scala/scala-dist/blob/2.13.x/scripts/jobs/release/website/update-api#L15
• Check that the API docs are published
  • http://www.scala-lang.org/api/$SCALA_VER/
  • https://docs.scala-lang.org/api/all.html ?
• Merge the scala-lang PR
  • wait for it to reach https://www.scala-lang.org and make sure it looks okay

Modules

• build and release scala-collection-compat and other modules (or open tickets asking that the maintainers do so)
  • this work has moved to https://github.com/scala/make-release-notes/blob/2.13.x/projects-2.13.md
• if it's a 2.12.x release, publish macro paradise for the new version

Announcements

• Scala Users discourse https://users.scala-lang.org
• add a reply on the same https://contributors.scala-lang.org thread where you announced that the release process was starting
• Tweet from @scala_lang
• https://gitter.im/scala/scala
  • if you feel like it, say something in https://gitter.im/scala/contributors too
• ask Seth to announce on #scala IRC

Afterwards

• Create PR to update scala-version on docs.scala-lang.org (example: bump scala-version docs.scala-lang#1294)
• Create PR to update versions.properties (new file generated by the bootstrap job) and the baseVersion in build.sbt
• Create PR to update https://github.com/lightbend/reactive-platform-docs/blob/master/src/main/paradox/supported-modules/index.md
• Update WhiteSource

[adriaanm]

Still deciding whether we need an RC3, but in the meantime...

[adriaanm]

A few small tweaks to consider:

1. new immutable.HashSet() no longer works (add such constructor).
2. FutureConverters: use asJava, not toJava, as they wrap.
3. maybe look again at the API changes of Option? (Make Option extend IterableOnce scala#8038 (comment)) (edited)
4. A default implementation of knownSize in IterableOnce rather than IterableOnceOps
5. Rename keyStepper and valueStepper in Map to mirror keysIterator and valuesIterator

[adriaanm]

Plus the PRs already scheduled: https://github.com/scala/scala/milestone/80

[Philippus]

Does that mean scala/scala#8063 is back on the table? as per scala/scala#8063 (comment)

[adriaanm]

Perhaps. We're still contemplating whether we even want to do an RC3. Once we do, we'll still be extremely conservative about what goes in, because the RC cycle has to converge sooner rather than later...

[adriaanm]

To be a bit more concrete: I'm warming up to doing an RC3, but I want the rule of thumb to be that a PR has to be an extremely clear win. There are two kinds of changes we can still make, in descending priority:

1. fixes for critical regressions (i.e. used to work in 2.12, now broken and no clear workaround): this will block a release and we will delay accordingly: the only PR that comes close to meeting this requirement is Remove unnecessary type parameter on IterableOnce#stepper scala#8083
2. small improvements that are clear wins but that couldn't land in 2.13.0 due to binary compatibility issues; these are opportunistic inclusions triggered by the need for (1)-level changes. We cannot afford to hold up the release for them, but they would be nice to have.

Let's be very critical in assigning even (2) status to the PRs currently in queue. I really appreciate everyone's drive for perfection and the final polish -- I really do -- but now is the time to grit our teeth and accept that 2.13.0 is as good as we can make it, and by that I mean very, very good. It's going to be an outstanding release. Now we just need to get it to the final :-)

[adriaanm]

As a separate comment, because timing is everything right now: if we do an RC3 (and I still need to run it by the team), it would have to be tomorrow Wed May 29th for us to get the final out before Scala Days. I hope we can all agree the latter's a suitable hard deadline :-)

[lrytz]

Does that mean scala/scala#8063 is back on the table

We'd need to have a PR very quickly to consider it. And it should be done for both Double and Float.

[lrytz]

Here's the list of methods that would need to be implemented if Option extended IterableOnceOps[A, IterableOnce, IterableOnce[A]]:

  def scanLeft[B](z: B)(op: (B, A) => B): IterableOnce[B] = ???
  def take(n: Int): Option[A] = ???
  def takeWhile(p: A => Boolean): Option[A] = ???
  def drop(n: Int): Option[A] = ???
  def dropWhile(p: A => Boolean): Option[A] = ???
  def slice(from: Int, until: Int): Option[A] = ???
  def flatMap[B](f: A => IterableOnce[B]): IterableOnce[B] = ???
  def flatten[B](implicit asIterable: A => IterableOnce[B]): IterableOnce[B] = ???
  def zipWithIndex: Option[(A, Int)] = ???
  def span(p: A => Boolean): (Option[A], Option[A]) = ???
  def tapEach[U](f: A => U): Option[A] = ???

[lrytz]

And here's the list of methods that Option would acquire by extending IterableOnceOps: https://gist.github.com/lrytz/4900473bba1640dda9d7a52aa3c0490d/revisions?diff=unified#diff-03c850668807a029aa19d81bfc077d9d

[lrytz]

If I remember correctly, our argument for making Option extend IterableOnce, but not IterableOnceOps, was that it fixes the issue we were discussing (scala/bug#11033) without pulling a large number of new methods into Option (see gist link above). This leaves the door open to revert that change if we ever can / want to, for example if type inference for overloaded methods is improved to fix the issue directly.

I think we should not extend IterableOnceOps, it's a really large interface. Maybe we should re-consider the deprecation of option2iterable? This would be the least-breaking change. We could deprecate it in 2.14, but make sure to add methods that are commonly used on Option via option2iterable and missing in Option today.