Merge pull request #913 from jvican/topic/bsp

Add bsp blog post

Author: jvican

blog/_posts/2018-06-10-bsp.md

@@ -0,0 +1,240 @@
+---
+layout: blog-detail
+post-type: blog
+by: Jorge Vicente Cantero
+title: "Announcing the Build Server Protocol v1.0.0"
+---
+
+I've always been convinced that the choice of an editor, IDE or build tool is
+personal and, therefore, asking you to go out of your way to learn a new tool
+to write Scala is not a viable solution. First, because changing people's
+habits is difficult and takes time. Second, it frustrates beginners who just
+want to learn the language. Last, it distracts seasoned developers from
+writing good Scala code until they master their toolchain. In the worst case,
+people will give up and not learn the language because the "cognitive
+overhead" would be too high.
+
+Instead it is much better if we, tooling developers, can bear the burden of
+shipping tools that integrate with your existing workflow and you, as a user,
+can just install them with a quick search and a single click.
+
+Today, I introduce the Build Server Protocol, a project that I believe plays
+a fundamental role in bringing us closer to making this vision a reality.
+
+## Any IDE, Any Build Tool
+
+Traditionally, every IDE has implemented a custom integration for each
+supported build tool in order to extract build information such as classpaths
+or source directories for every project in your build. This information is
+necessary for the IDE to present you with a workspace where you can browse,
+write and compile Scala code. Every time you "Import Project" from IntelliJ
+or Eclipse, the IDE carries out this process on the background.
+
+Likewise, all build tools are expected to integrate with all available IDEs.
+Most of them allow you to export their build to an external project model so
+that user adoption is not hurt by lack of official IDE support. This support
+isn't as thorough and polished as the one provided by IDEs, but usually gets
+the job done at the expense of supported features.
+
+So, IDEs create their own custom build tool integrations and build tools
+provide their own custom IDE integrations. Why should you care?
+
+It turns out the growing number of language servers and build tools in the
+wider programming community means tooling developers spend a lot of time
+working on these integrations. This work, that needs to be duplicated on each
+side every time there is a new IDE or build tool and distracts developers
+from working on bugfixes and performance improvements, ends up affecting
+us -- their end users. This is a big deal; we developers rely on build tools and
+IDEs for everything! If they are slow to evolve and maintain, then our
+developer experience takes a hit.
+
+The idea behind the Build Server Protocol is to standardize the protocol for
+how build tools (servers) and IDEs (clients) communicate, so a single Build
+Server can be re-used in multiple IDEs, and IDEs can support build tools with
+minimal effort.
+
+As a result, end users are provided with the best developer experience and
+tooling developers can improve the quality of their integrations with less
+effort and time, without locking out a part of the programming language
+community from good language support. This is especially important for the
+Scala community that has had a lot of recent activity in the build tools
+space with the increasing adoption of build tools like [Pants], [Bazel] or
+[mill].
+
+### Relationship with the Language Server Protocol
+
+BSP is a win for both IDEs implementors, build tool vendors and their users!
+But this is, in fact, not the first time this idea is applied to tools.
+Microsoft's [Language Server Protocol][LSP] (LSP) applied the same concept to
+language servers and editors. What is then BSP's relationship with LSP?
+
+BSP and LSP are complementary. While LSP specifies endpoints for
+communication between an *editor acting as a client* and a language server,
+BSP specifies endpoints between a *language server acting as a client* and a
+build server.
+
+It turns out that if we want to create language servers to support Scala in
+editors like Visual Studio Code, `vim` or Sublime, we also need to integrate
+with build tools to support features like compiler diagnostics or code
+navigation. One of the main properties of BSP is that it can be implemented
+alongside LSP.
+
+## The Build Server Protocol v1.0.0
+
+Today I'd like to announce `v1.0.0` of the Build Server Protocol. This
+version has been the result of a fruitful collaboration between the Scala
+Center (me and [Ólafur Páll Geirsson][olafurpg]) and [Justin
+Kaeser][jastice], the [JetBrains] developer responsible for sbt support in
+[IntelliJ IDEA].
+
+Justin and I have spent the past months implementing a prototype of the Build
+Server Protocol between [IntelliJ IDEA] (client) and [Bloop] (server). Bloop
+is a build-tool-agnostic compilation server that integrates with build tools
+such as [sbt], [mill] or [Maven] to allow you [to compile, test and run your
+applications much faster and outside of your stock build
+tool](https://www.scala-lang.org/blog/2017/11/30/bloop-release.html). Bloop
+has a client-server architecture, so it is the perfect project to implement
+BSP.
+
+The goal of this prototype was two-fold. On the one hand, we wanted to
+experience first-hand the challenges of implementing BSP and learn new
+insights to improve the protocol. On the other hand, we wanted to provide
+Scala developers a faster "Import Project" that would work across different
+build tools *now*, even though build tools don't yet provide implementations
+of build servers.
+
+[The first version of
+BSP](https://github.com/scalacenter/bsp/blob/master/docs/bsp.md) has been
+designed to support the most basic language integration in an IDE/editor.
+IDEs can find all the modules defined in a workspace, ask for their
+dependencies, compile, test, run, and get notifications on modified build
+targets, among a few other features.
+
+Our IntelliJ-Bloop implementation doesn't yet implement all these endpoints.
+Instead, it focuses on providing a faster integration than the traditional
+project import. You can now import your project, browse Scala code with full
+language support and get compiler diagnostics from the build tool in
+IntelliJ's editor.
+
+The process to import project via BSP is as follows:
+
+1. [Export your build to Bloop][bloop-instructions]. The Bloop server will
+   immediately pick up these changes.
+2. Import the build from Bloop to your IDE via BSP.
+
+<p></p><!-- To give more space between list and next paragraph. -->
+
+In total, this two-step process takes up to **15 seconds** in a medium-sized
+build where all the dependencies have already been fetched. It is already
+available in IntelliJ EAP 2018.2 under a experimental flag. To try the
+prototype out, read [the full instructions to get Bloop and IntelliJ set
+up](https://gist.github.com/jastice/212227aa5fc29426aeea3b93280bcd8a).
+
+You can see it on this demo importing [akka/akka](https://github.com/akka/akka/):
+
+![Demo of BSP import for akka/akka](/resources/img/bloop-install.gif)
+
+The two-step process is fast, but it currently requires too much user
+intervention and needs to be repeated every time something changes in the
+build. What can we do to improve it?
+
+Bloop could do a better job at exporting your sbt build every time you start
+up your sbt shell or `reload` your build to pick up new changes. This would
+remove the need for the first step. But the process is still ardous for
+users; they need to still perform the manual step of reloading the project
+every time the build tool and IDE go out of sync.
+
+BSP has been designed to deal with this issue from the start. The fact that
+it encourages a bidirectional communication between the build tool and the
+IDE means that build tools can send notifications to IDEs whenever there is a
+change in your build. What's most interesting is that a future release of a
+build tool supporting BSP would suffice to provide this state-of-the-art
+project import experience instantly.
+
+Aside from importing your build, the BSP prototype also allows you to get
+compiler diagnostics directly from your build tool instead of diagnostics
+from IntelliJ's presentation compiler. Build tool diagnostics are always
+correct since the compilation happens in your build tool and the results
+are streamed to the IDE.
+
+![Demo of BSP compile for akka/akka](/resources/img/bloop-compile.gif)
+
+
+To learn more about the Build Server Protocol, check out the
+[slides](http://jorge.vican.me/slides/BSP.pdf) of my talk with Justin at
+Scalasphere 2018.
+
+## Next steps
+
+The next steps will focus on increasing adoption of the protocol and
+enhancing the availability and quality of the integrations for users.
+
+First, the Scala Center will keep collaborating with Jetbrains to improve
+IntelliJ's BSP import for Bloop and other build tools. The goal is to bring
+BSP import to a production-ready level where users can benefit from a
+build-tool-agnostic import project that is both faster and more reliable.
+
+Second, I'd like to invest some resources in creating a better synergy
+between build tool authors and IDE implementors. The [Scala Tooling Protocols
+Working Group](https://github.com/scalacenter/tooling-working-groups/)
+created by [Jon
+Pretty](https://www.scala-lang.org/blog/2018/02/14/tooling.html) a few months
+ago will help us discuss and refine the protocol so that all tooling
+developers in our community can implement the protocol at scale and gain more
+experience from this process.
+
+Third, I'd like to collaborate more closely with developers working on Scala
+language servers like [Metals] or [Dotty IDE] to bring Scala support to all
+editors. There is a big chunk of Scala developers that don't use IntelliJ and
+I'd like to make sure that they benefit from a smooth developer experience
+too.
+
+
+Last but not least, I'd like to reach out to folks in other programming
+language communities to find out ways we can pull our efforts together.
+The Build Server Protocol is **a language-agnostic protocol that can be
+modified to add support for any programming language** and I would love to
+see other communities improving the future of the build tools and IDE
+communication.
+
+## How to get involved
+
+The best way to share your thoughts on the Build Server Protocol or to get
+involved in its development is to open an issue or pull request to
+[`scalacenter/bsp`](https://github.com/scalacenter/bsp). Most of the features
+in the specification have tickets and review discussions elaborating on the
+design goals, like these [closed](https://github.com/scalacenter/bsp/issues/3)
+and [open](https://github.com/scalacenter/bsp/issues/14) tickets.
+
+If you want to help integrating BSP in your favorite editor or IDE, or you'd
+like to help test the integrations, join our
+[`scalacenter/bsp`](https://gitter.im/scalacenter/bsp) Gitter channel and
+let's discuss ways we can work together.
+
+## Acknowledgements
+
+I'd like to thank my colleague at the Scala Center, Ólafur Páll Geirsson, for
+being the co-author of the Build Server Protocol. His help and insights, as
+well as his detailed understanding of [LSP], have been fundamental in
+designing BSP and focusing on the ergonomics of the clients (IDEs) first.
+
+Finally, I'd like to thank Justin Kaeser and, more broadly, JetBrains for
+being part of this initiative. I am optimistic our collaboration can lead to
+better and faster developer tools for the broad of the Scala community.
+
+[sbt]: https://www.scala-sbt.org/
+[Maven]: http://maven.apache.org/
+[Bazel]: https://bazel.build
+[Pants]: https://pantsbuild.org
+[mill]: https://lihaoyi.com/mill
+[langservers]: http://langserver.org
+[LSP]: https://microsoft.github.io/language-server-protocol/specification
+[JetBrains]: https://www.jetbrains.com/
+[IntelliJ IDEA]: https://www.jetbrains.com/idea/
+[jastice]: https://github.com/jastice
+[coursier]: https://github.com/coursier/coursier
+[Bloop]: https://github.com/scalacenter/bloop
+[Metals]: https://github.com/scalameta/metals
+[Dotty IDE]: https://dotty.epfl.ch/docs/usage/ide-support.html
+[olafurpg]: https://github.com/olafurpg
+[bloop-instructions]: https://scalacenter.github.io/bloop/docs/installation/#generate-configuration-files-for-your-project