|
| 1 | +--- |
| 2 | +layout: blog |
| 3 | +post-type: blog |
| 4 | +by: Ólafur Páll Geirsson |
| 5 | +title: "Introducing Scalafix: a migration tool for Scalac to Dotty" |
| 6 | +--- |
| 7 | + |
| 8 | +I am excited to announce the first release of |
| 9 | +[scalafix](http://scalacenter.github.io/scalafix/), a new tool to |
| 10 | +prepare Scala 2.x code for [Dotty](http://dotty.epfl.ch/), a next |
| 11 | +generation Scala compiler. This effort follows the Scala Center Advisory |
| 12 | +Board proposal: |
| 13 | +["Clarification of Scala to Dotty migration path"](https://scala.epfl.ch/minutes/2016/06/06/may-9-2016.html#proposal-scp-002-clarification-of-scala-to-dotty-migration-path). |
| 14 | + |
| 15 | +There is a lot to be excited over in Dotty, |
| 16 | +[faster compilation times](http://www.slideshare.net/Odersky/scala-days-nyc-2016), |
| 17 | +[faster+smaller binaries](https://d-d.me/talks/scaladays2015/#/) and |
| 18 | +[awesome error messages](http://www.scala-lang.org/blog/2016/10/14/dotty-errors.html) |
| 19 | +to name a few things. However, some Scala 2.x applications can't |
| 20 | +immediately enjoy these benefits due to several breaking changes in |
| 21 | +Dotty. Scalafix is part of our strategy to smoothen the migration |
| 22 | +process for those applications. |
| 23 | + |
| 24 | +Scalafix takes care of easy, repetitive and tedious code transformations |
| 25 | +so you can focus on the changes that truly deserve your attention. In a |
| 26 | +nutshell, scalafix reads a source file, transforms usage of unsupported |
| 27 | +features into newer alternatives, and writes the final result back to |
| 28 | +the original source file. Scalafix aims to automate the migration of as |
| 29 | +many unsupported features as possible. There will be cases where |
| 30 | +scalafix is unable to perform automatic migration. In such situations, |
| 31 | +scalafix will point to the offending source line and provide |
| 32 | +instructions on how to refactor the code. The objective is that your |
| 33 | +Scala 2.x codebase gets up and running faster with Dotty. |
| 34 | + |
| 35 | +What can scalafix do? |
| 36 | +------------------------- |
| 37 | + |
| 38 | +Scalafix comes with a command line interface and an SBT plugin. Running |
| 39 | +scalafix is as simple as: |
| 40 | + |
| 41 | +~~~ |
| 42 | +# CLI |
| 43 | +scalafix Code.scala |
| 44 | +# SBT, after adding plugin to project/plugins.sbt |
| 45 | +sbt scalafix |
| 46 | +~~~ |
| 47 | + |
| 48 | +More detailed instructions on how to setup scalafix are on the |
| 49 | +[project website](http://scalacenter.github.io/scalafix/). |
| 50 | + |
| 51 | +This initial release implements two rewrites: `ProcedureSyntax` and |
| 52 | +`VolatileLazyVal`. More rewrite rules are planned for future releases. |
| 53 | + |
| 54 | +The `ProcedureSyntax` rewrite works like this: |
| 55 | + |
| 56 | +~~~scala |
| 57 | +// before |
| 58 | +def main(args: Seq[String]) { // note lack of '=' |
| 59 | + println("Hello scalafix!") |
| 60 | +} |
| 61 | +// after |
| 62 | +def main(args: Seq[String]): Unit = { // note ': Unit =' |
| 63 | + println("Hello scalafix!") |
| 64 | +} |
| 65 | +~~~ |
| 66 | + |
| 67 | +Dropping procedure syntax is part of a general effort in Dotty to |
| 68 | +simplify the Scala language. |
| 69 | + |
| 70 | +The `VolatileLazyVal` rewrite adds a `@volatile` annotation to lazy vals, |
| 71 | +like this: |
| 72 | + |
| 73 | +~~~scala |
| 74 | +lazy val x = ... // before |
| 75 | +@volatile lazy val x = ... // after |
| 76 | +~~~ |
| 77 | + |
| 78 | +With `@volatile`, Dotty uses a deadlock free scheme that is |
| 79 | +comparable-if-not-faster than the scheme used in scalac. |
| 80 | + |
| 81 | +How does scalafix work? |
| 82 | +--------------------------- |
| 83 | + |
| 84 | +All the heavy-lifting behind scalafix happens in |
| 85 | +[scala.meta](http://scalameta.org/), a new metaprogramming toolkit |
| 86 | +aspiring to succeed scala.reflect. Scala.meta makes it easy to do |
| 87 | +non-trivial code transformations while preserving syntactic details in |
| 88 | +the original source file. This key attribute of scala.meta makes it |
| 89 | +suitable for both developer tools like scalafix as well as compile-time |
| 90 | +metaprograms like macros. |
| 91 | + |
| 92 | +The grand vision of scala.meta is to provide a single interface to |
| 93 | +accomplish most common metaprogramming tasks in |
| 94 | +Scala. |
| 95 | + |
| 96 | + The idea is to untie metaprograms from compiler implementations, providing a |
| 97 | +platform-independent API that works both in the Scala compiler, Dotty |
| 98 | +and tools like IntelliJ IDEA. Metaprogram authors benefit from a |
| 99 | +user-friendly interface built on immutable ADTs and type classes. |
| 100 | +Compiler authors benefit from the fact that they don't need to expose |
| 101 | +compiler internals in order to support popular features like macros. |
| 102 | + |
| 103 | +In scalafix, we use the scala.meta API to parse Scala source files, |
| 104 | +traverse abstract syntax trees, inspect tokens (including comments!) and |
| 105 | +then perform the minimum required transformations to the source file. |
| 106 | +Thanks to scala.meta dialects, which abstract away syntactic differences |
| 107 | +between variants of Scala, we can use the same programs to manipulate |
| 108 | +regular Scala source files, SBT files as well as code that uses new |
| 109 | +Dotty syntax such as union types and trait parameters. All this powerful |
| 110 | +functionality is implemented behind a minimalistic interface. |
| 111 | +The NonVolatileLazyVal |
| 112 | +[implementation](https://github.com/scalacenter/scalafix/blob/master/core/src/main/scala/scalafix/rewrite/VolatileLazyVal.scala) |
| 113 | +is 15 lines of code. It could comfortably fit in three tweets. |
| 114 | + |
| 115 | +We would not be able to achieve this level of expressiveness with |
| 116 | +scala.reflect, the current standard metaprogramming framework for Scala. |
| 117 | +Why? To name a few reasons, scala.reflect does not preserve syntactic |
| 118 | +details such as comments and scala.reflect has no notions of dialects, |
| 119 | +so it is bound to only support Scala 2.x. |
| 120 | + |
| 121 | +How does scalafix evolve? |
| 122 | +----------------------------- |
| 123 | + |
| 124 | +As we have seen above, the functionality of scalafix heavily relies on |
| 125 | +the infrastructure provided by scala.meta. However, in order to write |
| 126 | +more sophisticated scalafix rewrite rules, there are two main features |
| 127 | +missing in scala.meta, namely a *scalac converter* and *semantic API*. |
| 128 | +We are closely collaborating with Eugene Burmako, the lead developer of |
| 129 | +scala.meta, in the development efforts of these two features. |
| 130 | + |
| 131 | +### Scalac converter |
| 132 | + |
| 133 | +The scalac converter is a key subsystem of scala.meta. |
| 134 | +For every Scala feature, the converter recognizes its representation in the Scala |
| 135 | +compiler and translates it into the corresponding data structures of the |
| 136 | +scala.meta API. |
| 137 | + |
| 138 | +Over the last months, we’ve made great improvements to the converter, |
| 139 | +which is under development in the [scalameta/paradise](https://github.com/scalameta/paradise) repository. |
| 140 | +We have established a comprehensive test suite that consists of a sample of |
| 141 | +over 26.000 Scala source files collected from popular open-source |
| 142 | +projects. Thanks to the joint effort of the team of scala.meta |
| 143 | +[contributors](https://github.com/scalameta/paradise/graphs/contributors), |
| 144 | +the converter now supports about 19.000 source files form the test |
| 145 | +corpus. We are continuously working to increase that number, aiming to |
| 146 | +bring language coverage to 100%. |
| 147 | + |
| 148 | +### Semantic API |
| 149 | + |
| 150 | +The semantic API will enable scala.meta programs to inspect compiler information |
| 151 | +such as type inference, name resolution and other functionality that |
| 152 | +requires a compilation step. This opens opportunities for new scalafix rewrite |
| 153 | +rules that cannot be done on a purely syntactic level, |
| 154 | +like `NonVolatileLazyVal` and `ProcedureSyntax`. |
| 155 | + |
| 156 | +We already have a |
| 157 | +[working prototype](https://github.com/scalacenter/scalafix/pull/14) |
| 158 | +of the first scalafix rewrite that uses the semantic API: `ExplicitImplicit`. |
| 159 | +`ExplicitImplicit` inserts type annotations for implicit definitions, like this: |
| 160 | + |
| 161 | +~~~scala |
| 162 | +// before |
| 163 | +implicit val x = (_: String).length |
| 164 | +// after |
| 165 | +implicit val x: String => Int = (_: String).length |
| 166 | +~~~ |
| 167 | +`ExplicitImplicit` requires the semantic API in order to get the inferred type |
| 168 | +annotations. |
| 169 | + |
| 170 | +Towards new-style macros |
| 171 | +---------------------------- |
| 172 | + |
| 173 | +One could say that by contributing to scala.meta, we get two features |
| 174 | +for the price of one. First, an improved converter and semantic API |
| 175 | +enables us to implement more sophisticated scalafix rewrite rules. |
| 176 | +Secondly, we accelerate the development of a new macro system that will |
| 177 | +work both with scalac and Dotty. |
| 178 | + |
| 179 | +As announced at Scala Days 2016, the current macros based on |
| 180 | +scala.reflect will be going away. Such macros have a number of known |
| 181 | +issues, including over complicated API and lacking IDE support. More |
| 182 | +importantly, the deep coupling between old-style macros and scalac |
| 183 | +means macros written in the old system effectively cannot be |
| 184 | +ported to Dotty. The plan is to discontinue support for scala.reflect |
| 185 | +macros in favor of [a new macro system called |
| 186 | +inline/meta](https://github.com/scala/scala.github.com/pull/567). |
| 187 | + |
| 188 | +A technology preview of the new-style macros came out this summer. This |
| 189 | +early release is still somewhat rough around the edges. Nevertheless, |
| 190 | +[scalafmt](https://olafurpg.github.io/scalafmt/) is already using |
| 191 | +these new macro annotations in production for more than a month now. If you |
| 192 | +are interested in learning more, the best place to get started is |
| 193 | +[my recent Scala World workshop](http://olafurpg.github.io/scala.meta-workshop). |
| 194 | + |
| 195 | +Much like scalafix, new-style macros crucially rely on the |
| 196 | +infrastructure provided by scala.meta. Concretely, the converter and the |
| 197 | +semantic APIs constitute a significant chunk of the implementation |
| 198 | +effort behind the new macro system. |
| 199 | + |
| 200 | +Interestingly, our contributions to scala.meta were originally motivated |
| 201 | +by the needs of scalafix. However, they are also immediately helping the |
| 202 | +development of this new universal macro system for Scala. We are excited |
| 203 | +that our open-source collaboration with the scala.meta team brings |
| 204 | +multiple benefits for the good of everyone! |
| 205 | + |
| 206 | +Get involved |
| 207 | +---------------- |
| 208 | + |
| 209 | +Are you interested in metaprogramming, developer tools and running |
| 210 | +experiments on millions of lines of Scala code? Come chat with us on |
| 211 | +[Gitter][gitter], and we’ll discuss how you can make a difference in shaping the |
| 212 | +developer tools of the future! |
| 213 | + |
| 214 | +[gitter]: https://gitter.im/scalacenter/scalafix |
| 215 | + |
0 commit comments