|
| 1 | +--- |
| 2 | +layout: blog-detail |
| 3 | +post-type: blog |
| 4 | +by: Jorge Vicente Cantero |
| 5 | +title: "Announcing the Build Server Protocol v1.0.0" |
| 6 | +--- |
| 7 | + |
| 8 | +I've always been convinced that the choice of an editor, IDE or build tool is |
| 9 | +personal and, therefore, asking you to go out of your way to learn a new tool |
| 10 | +to write Scala is not a viable solution. First, because changing people's |
| 11 | +habits is difficult and takes time. Second, it frustrates beginners who just |
| 12 | +want to learn the language. Last, it distracts seasoned developers from |
| 13 | +writing good Scala code until they master their toolchain. In the worst case, |
| 14 | +people will give up and not learn the language because the "cognitive |
| 15 | +overhead" would be too high. |
| 16 | + |
| 17 | +Instead it is much better if we, tooling developers, can bear the burden of |
| 18 | +shipping tools that integrate with your existing workflow and you, as a user, |
| 19 | +can just install them with a quick search and a single click. |
| 20 | + |
| 21 | +Today, I introduce the Build Server Protocol, a project that I believe plays |
| 22 | +a fundamental role in bringing us closer to making this vision a reality. |
| 23 | + |
| 24 | +## Any IDE, Any Build Tool |
| 25 | + |
| 26 | +Traditionally, every IDE has implemented a custom integration for each |
| 27 | +supported build tool in order to extract build information such as classpaths |
| 28 | +or source directories for every project in your build. This information is |
| 29 | +necessary for the IDE to present you with a workspace where you can browse, |
| 30 | +write and compile Scala code. Every time you "Import Project" from IntelliJ |
| 31 | +or Eclipse, the IDE carries out this process on the background. |
| 32 | + |
| 33 | +Likewise, all build tools are expected to integrate with all available IDEs. |
| 34 | +Most of them allow you to export their build to an external project model so |
| 35 | +that user adoption is not hurt by lack of official IDE support. This support |
| 36 | +isn't as thorough and polished as the one provided by IDEs, but usually gets |
| 37 | +the job done at the expense of supported features. |
| 38 | + |
| 39 | +So, IDEs create their own custom build tool integrations and build tools |
| 40 | +provide their own custom IDE integrations. Why should you care? |
| 41 | + |
| 42 | +It turns out the growing number of language servers and build tools in the |
| 43 | +wider programming community means tooling developers spend a lot of time |
| 44 | +working on these integrations. This work, that needs to be duplicated on each |
| 45 | +side every time there is a new IDE or build tool and distracts developers |
| 46 | +from working on bugfixes and performance improvements, ends up affecting |
| 47 | +us -- their end users. This is a big deal; we developers rely on build tools and |
| 48 | +IDEs for everything! If they are slow to evolve and maintain, then our |
| 49 | +developer experience takes a hit. |
| 50 | + |
| 51 | +The idea behind the Build Server Protocol is to standardize the protocol for |
| 52 | +how build tools (servers) and IDEs (clients) communicate, so a single Build |
| 53 | +Server can be re-used in multiple IDEs, and IDEs can support build tools with |
| 54 | +minimal effort. |
| 55 | + |
| 56 | +As a result, end users are provided with the best developer experience and |
| 57 | +tooling developers can improve the quality of their integrations with less |
| 58 | +effort and time, without locking out a part of the programming language |
| 59 | +community from good language support. This is especially important for the |
| 60 | +Scala community that has had a lot of recent activity in the build tools |
| 61 | +space with the increasing adoption of build tools like [Pants], [Bazel] or |
| 62 | +[mill]. |
| 63 | + |
| 64 | +### Relationship with the Language Server Protocol |
| 65 | + |
| 66 | +BSP is a win for both IDEs implementors, build tool vendors and their users! |
| 67 | +But this is, in fact, not the first time this idea is applied to tools. |
| 68 | +Microsoft's [Language Server Protocol][LSP] (LSP) applied the same concept to |
| 69 | +language servers and editors. What is then BSP's relationship with LSP? |
| 70 | + |
| 71 | +BSP and LSP are complementary. While LSP specifies endpoints for |
| 72 | +communication between an *editor acting as a client* and a language server, |
| 73 | +BSP specifies endpoints between a *language server acting as a client* and a |
| 74 | +build server. |
| 75 | + |
| 76 | +It turns out that if we want to create language servers to support Scala in |
| 77 | +editors like Visual Studio Code, `vim` or Sublime, we also need to integrate |
| 78 | +with build tools to support features like compiler diagnostics or code |
| 79 | +navigation. One of the main properties of BSP is that it can be implemented |
| 80 | +alongside LSP. |
| 81 | + |
| 82 | +## The Build Server Protocol v1.0.0 |
| 83 | + |
| 84 | +Today I'd like to announce `v1.0.0` of the Build Server Protocol. This |
| 85 | +version has been the result of a fruitful collaboration between the Scala |
| 86 | +Center (me and [Ólafur Páll Geirsson][olafurpg]) and [Justin |
| 87 | +Kaeser][jastice], the [JetBrains] developer responsible for sbt support in |
| 88 | +[IntelliJ IDEA]. |
| 89 | + |
| 90 | +Justin and I have spent the past months implementing a prototype of the Build |
| 91 | +Server Protocol between [IntelliJ IDEA] (client) and [Bloop] (server). Bloop |
| 92 | +is a build-tool-agnostic compilation server that integrates with build tools |
| 93 | +such as [sbt], [mill] or [Maven] to allow you [to compile, test and run your |
| 94 | +applications much faster and outside of your stock build |
| 95 | +tool](https://www.scala-lang.org/blog/2017/11/30/bloop-release.html). Bloop |
| 96 | +has a client-server architecture, so it is the perfect project to implement |
| 97 | +BSP. |
| 98 | + |
| 99 | +The goal of this prototype was two-fold. On the one hand, we wanted to |
| 100 | +experience first-hand the challenges of implementing BSP and learn new |
| 101 | +insights to improve the protocol. On the other hand, we wanted to provide |
| 102 | +Scala developers a faster "Import Project" that would work across different |
| 103 | +build tools *now*, even though build tools don't yet provide implementations |
| 104 | +of build servers. |
| 105 | + |
| 106 | +[The first version of |
| 107 | +BSP](https://github.com/scalacenter/bsp/blob/master/docs/bsp.md) has been |
| 108 | +designed to support the most basic language integration in an IDE/editor. |
| 109 | +IDEs can find all the modules defined in a workspace, ask for their |
| 110 | +dependencies, compile, test, run, and get notifications on modified build |
| 111 | +targets, among a few other features. |
| 112 | + |
| 113 | +Our IntelliJ-Bloop implementation doesn't yet implement all these endpoints. |
| 114 | +Instead, it focuses on providing a faster integration than the traditional |
| 115 | +project import. You can now import your project, browse Scala code with full |
| 116 | +language support and get compiler diagnostics from the build tool in |
| 117 | +IntelliJ's editor. |
| 118 | + |
| 119 | +The process to import project via BSP is as follows: |
| 120 | + |
| 121 | +1. [Export your build to Bloop][bloop-instructions]. The Bloop server will |
| 122 | + immediately pick up these changes. |
| 123 | +2. Import the build from Bloop to your IDE via BSP. |
| 124 | + |
| 125 | +<p></p><!-- To give more space between list and next paragraph. --> |
| 126 | + |
| 127 | +In total, this two-step process takes up to **15 seconds** in a medium-sized |
| 128 | +build where all the dependencies have already been fetched. It is already |
| 129 | +available in IntelliJ EAP 2018.2 under a experimental flag. To try the |
| 130 | +prototype out, read [the full instructions to get Bloop and IntelliJ set |
| 131 | +up](https://gist.github.com/jastice/212227aa5fc29426aeea3b93280bcd8a). |
| 132 | + |
| 133 | +You can see it on this demo importing [akka/akka](https://github.com/akka/akka/): |
| 134 | + |
| 135 | + |
| 136 | + |
| 137 | +The two-step process is fast, but it currently requires too much user |
| 138 | +intervention and needs to be repeated every time something changes in the |
| 139 | +build. What can we do to improve it? |
| 140 | + |
| 141 | +Bloop could do a better job at exporting your sbt build every time you start |
| 142 | +up your sbt shell or `reload` your build to pick up new changes. This would |
| 143 | +remove the need for the first step. But the process is still ardous for |
| 144 | +users; they need to still perform the manual step of reloading the project |
| 145 | +every time the build tool and IDE go out of sync. |
| 146 | + |
| 147 | +BSP has been designed to deal with this issue from the start. The fact that |
| 148 | +it encourages a bidirectional communication between the build tool and the |
| 149 | +IDE means that build tools can send notifications to IDEs whenever there is a |
| 150 | +change in your build. What's most interesting is that a future release of a |
| 151 | +build tool supporting BSP would suffice to provide this state-of-the-art |
| 152 | +project import experience instantly. |
| 153 | + |
| 154 | +Aside from importing your build, the BSP prototype also allows you to get |
| 155 | +compiler diagnostics directly from your build tool instead of diagnostics |
| 156 | +from IntelliJ's presentation compiler. Build tool diagnostics are always |
| 157 | +correct since the compilation happens in your build tool and the results |
| 158 | +are streamed to the IDE. |
| 159 | + |
| 160 | + |
| 161 | + |
| 162 | + |
| 163 | +To learn more about the Build Server Protocol, check out the |
| 164 | +[slides](http://jorge.vican.me/slides/BSP.pdf) of my talk with Justin at |
| 165 | +Scalasphere 2018. |
| 166 | + |
| 167 | +## Next steps |
| 168 | + |
| 169 | +The next steps will focus on increasing adoption of the protocol and |
| 170 | +enhancing the availability and quality of the integrations for users. |
| 171 | + |
| 172 | +First, the Scala Center will keep collaborating with Jetbrains to improve |
| 173 | +IntelliJ's BSP import for Bloop and other build tools. The goal is to bring |
| 174 | +BSP import to a production-ready level where users can benefit from a |
| 175 | +build-tool-agnostic import project that is both faster and more reliable. |
| 176 | + |
| 177 | +Second, I'd like to invest some resources in creating a better synergy |
| 178 | +between build tool authors and IDE implementors. The [Scala Tooling Protocols |
| 179 | +Working Group](https://github.com/scalacenter/tooling-working-groups/) |
| 180 | +created by [Jon |
| 181 | +Pretty](https://www.scala-lang.org/blog/2018/02/14/tooling.html) a few months |
| 182 | +ago will help us discuss and refine the protocol so that all tooling |
| 183 | +developers in our community can implement the protocol at scale and gain more |
| 184 | +experience from this process. |
| 185 | + |
| 186 | +Third, I'd like to collaborate more closely with developers working on Scala |
| 187 | +language servers like [Metals] or [Dotty IDE] to bring Scala support to all |
| 188 | +editors. There is a big chunk of Scala developers that don't use IntelliJ and |
| 189 | +I'd like to make sure that they benefit from a smooth developer experience |
| 190 | +too. |
| 191 | + |
| 192 | + |
| 193 | +Last but not least, I'd like to reach out to folks in other programming |
| 194 | +language communities to find out ways we can pull our efforts together. |
| 195 | +The Build Server Protocol is **a language-agnostic protocol that can be |
| 196 | +modified to add support for any programming language** and I would love to |
| 197 | +see other communities improving the future of the build tools and IDE |
| 198 | +communication. |
| 199 | + |
| 200 | +## How to get involved |
| 201 | + |
| 202 | +The best way to share your thoughts on the Build Server Protocol or to get |
| 203 | +involved in its development is to open an issue or pull request to |
| 204 | +[`scalacenter/bsp`](https://github.com/scalacenter/bsp). Most of the features |
| 205 | +in the specification have tickets and review discussions elaborating on the |
| 206 | +design goals, like these [closed](https://github.com/scalacenter/bsp/issues/3) |
| 207 | +and [open](https://github.com/scalacenter/bsp/issues/14) tickets. |
| 208 | + |
| 209 | +If you want to help integrating BSP in your favorite editor or IDE, or you'd |
| 210 | +like to help test the integrations, join our |
| 211 | +[`scalacenter/bsp`](https://gitter.im/scalacenter/bsp) Gitter channel and |
| 212 | +let's discuss ways we can work together. |
| 213 | + |
| 214 | +## Acknowledgements |
| 215 | + |
| 216 | +I'd like to thank my colleague at the Scala Center, Ólafur Páll Geirsson, for |
| 217 | +being the co-author of the Build Server Protocol. His help and insights, as |
| 218 | +well as his detailed understanding of [LSP], have been fundamental in |
| 219 | +designing BSP and focusing on the ergonomics of the clients (IDEs) first. |
| 220 | + |
| 221 | +Finally, I'd like to thank Justin Kaeser and, more broadly, JetBrains for |
| 222 | +being part of this initiative. I am optimistic our collaboration can lead to |
| 223 | +better and faster developer tools for the broad of the Scala community. |
| 224 | + |
| 225 | +[sbt]: https://www.scala-sbt.org/ |
| 226 | +[Maven]: http://maven.apache.org/ |
| 227 | +[Bazel]: https://bazel.build |
| 228 | +[Pants]: https://pantsbuild.org |
| 229 | +[mill]: https://lihaoyi.com/mill |
| 230 | +[langservers]: http://langserver.org |
| 231 | +[LSP]: https://microsoft.github.io/language-server-protocol/specification |
| 232 | +[JetBrains]: https://www.jetbrains.com/ |
| 233 | +[IntelliJ IDEA]: https://www.jetbrains.com/idea/ |
| 234 | +[jastice]: https://github.com/jastice |
| 235 | +[coursier]: https://github.com/coursier/coursier |
| 236 | +[Bloop]: https://github.com/scalacenter/bloop |
| 237 | +[Metals]: https://github.com/scalameta/metals |
| 238 | +[Dotty IDE]: https://dotty.epfl.ch/docs/usage/ide-support.html |
| 239 | +[olafurpg]: https://github.com/olafurpg |
| 240 | +[bloop-instructions]: https://scalacenter.github.io/bloop/docs/installation/#generate-configuration-files-for-your-project |
0 commit comments