IDE support for Dotty via the Language Server Protocol, including a Visual Studio Code extension #2532

smarter · 2017-05-25T17:24:45Z

This PR introduces IDE support to Dotty, this requires several components (see each individual commit messages for more detailed explanations):

A set of experimental APIs in the compiler designed for IDE usage: dotty.tools.dotc.interactive.*. These APIs were developed for the Dotty Language Server but could evolve to support other usages like the REPL or other IDEs.
An implementation of the Language Server Protocol for Dotty, this allow any editor supporting the LSP to get great Dotty support with a minimal amount of work (see http://langserver.org/ for a list of LSP implementations).
A Visual Studio Code extension to run the Dotty Language Server.
New commands in the sbt-dotty plugin to setup everything needed to run the Dotty Language Server and start Visual Studio Code.

Quick start

Install Visual Studio Code
In your own project compiled with dotty, make sure sbt-dotty 0.1.0-RC5 is used, and run:

sbt launchIDE

Status

Fully supported features:

Typechecking as you type to show compiler errors/warnings
Type information on hover
Go to definition (in the current project)
Find all references

Partially working features:

Completion
Renaming
Go to definition in external projects

Unimplemented features:

Documentation on hover
Formatting code (requires integrating with scalafmt)
Quick fixes (probably by integrating with scalafix)

Current limitations, to be fixed:

Projects should be compiled with sbt before starting the IDE, this is
automatically done for you if you run sbt launchIDE.
Once the IDE is started, source files that are not opened in the IDE
should not be modified in some other editor, the IDE won't pick up
these changes.
Not all compiler errors/warnings are displayed, just those occuring
during typechecking.

Future

There is much more that could be done, in particular see https://github.com/lampepfl/dotty/issues/1523 for a lot of potential improvements that we should keep in mind.

smarter · 2017-05-25T18:29:35Z

/cc @fommil @dragos @sschaef

fommil · 2017-05-26T04:01:48Z

nice! btw, we're going to add language server to ensime. This will serve as an excellent reference implementation for ensime 3.0. Hmm, I guess scastie might need to use this for dotty support.

felixmulder

I see that cancelToken is not used anywhere, I assume we'd need to implement safe points in the compiler to use that.

Let's remember to open an issue for that once this is merged.

Otherwise LGTM, happy to have this in!

felixmulder · 2017-05-26T07:38:38Z

compiler/src/dotty/tools/dotc/core/SymbolLoaders.scala

+        classRoot.symbol.asClass.unpickler = unpickler
+        moduleRoot.symbol.asClass.unpickler = unpickler
+      case _ =>
+    }


What was the reason behind making this non-RT?

What do you mean by RT here? Referentially transparent? This method is never called multiple times on the same denotation.

felixmulder · 2017-05-26T07:46:33Z

compiler/src/dotty/tools/dotc/interactive/Interactive.scala

+   */
+  @tailrec def sourceSymbol(sym: Symbol)(implicit ctx: Context): Symbol =
+    if (!sym.exists)
+      sym


=> NoSymbol ?

That's completely equivalent, do you want me to change this for style reasons?

I'll leave that up to you.

I think either works fine.

felixmulder · 2017-05-26T07:53:13Z

compiler/src/dotty/tools/dotc/interactive/InteractiveDriver.scala

+
+  private var myCtx: Context = myInitCtx
+
+  def currentCtx: Context = myCtx


Does this need to be public?

Yes, very much so, see usage in DottyLanguageServer

felixmulder · 2017-05-26T07:55:19Z

compiler/src/dotty/tools/dotc/interactive/InteractiveDriver.scala

+  // Like in `ZipArchiveFileLookup` we assume that zips are immutable
+  private val zipClassPathClasses: Seq[String] = zipClassPaths.flatMap { zipCp =>
+    // (new ZipFile(zipCp.asInstanceOf[ZipArchiveFileLookup[_]].zipFile).stream.collect(Collectors.toList()).asScala: List[ZipEntry])
+    // Working with Java 8 stream without SAMs and scala-java8-compat is awful.


felixmulder · 2017-05-26T07:58:14Z

compiler/src/dotty/tools/dotc/interactive/InteractiveDriver.scala

+  private val myOpenedTrees = new mutable.LinkedHashMap[URI, List[SourceTree]]
+
+  def openedFiles: Map[URI, SourceFile] = myOpenedFiles
+  def openedTrees: Map[URI, List[SourceTree]] = myOpenedTrees


Yes, see usage in DottyLanguageServer

felixmulder · 2017-05-26T08:11:20Z

language-server/src/dotty/tools/languageserver/DottyLanguageServer.scala

+  }
+
+  override def didChangeConfiguration(params: DidChangeConfigurationParams): Unit =
+    /*thisServer.synchronized*/ {}


Maybe a docstring here?

A docstring for what? If you're curious what didChangeConfiguration is for, see the spec: https://github.com/Microsoft/language-server-protocol/blob/master/protocol.md#didchangeconfiguration-notification
The commented synchronized is so that if we ever implement this method, we don't forget to make it synchronized

felixmulder · 2017-05-26T08:11:25Z

language-server/src/dotty/tools/languageserver/DottyLanguageServer.scala

+    /*thisServer.synchronized*/ {}
+
+  override def didChangeWatchedFiles(params: DidChangeWatchedFilesParams): Unit =
+    /*thisServer.synchronized*/ {}


felixmulder · 2017-05-26T08:11:29Z

language-server/src/dotty/tools/languageserver/DottyLanguageServer.scala

+    /*thisServer.synchronized*/ {}
+
+  override def didSave(params: DidSaveTextDocumentParams): Unit =
+    /*thisServer.synchronized*/ {}


felixmulder · 2017-05-26T08:14:21Z

language-server/src/dotty/tools/languageserver/Main.scala

+/** Run the Dotty Language Server.
+ *
+ * This is designed to be started from an editor supporting the Language Server
+ * Protocol, the easiest way to fetch and run this is to use `coursier`:


felixmulder · 2017-05-26T08:14:38Z

language-server/src/dotty/tools/languageserver/Main.scala

+ * This is designed to be started from an editor supporting the Language Server
+ * Protocol, the easiest way to fetch and run this is to use `coursier`:
+ *
+ *   coursier launch $artifact -M dotty.tools.languageserver.Main -- -stdio


I tried it, and it doesn't do any highlighting on this line, so no point in using this syntax.

smarter

Thanks for the review!

smarter · 2017-05-26T12:31:51Z

compiler/src/dotty/tools/dotc/core/SymbolLoaders.scala

+        classRoot.symbol.asClass.unpickler = unpickler
+        moduleRoot.symbol.asClass.unpickler = unpickler
+      case _ =>
+    }


What do you mean by RT here? Referentially transparent? This method is never called multiple times on the same denotation.

smarter · 2017-05-26T12:33:08Z

compiler/src/dotty/tools/dotc/interactive/Interactive.scala

+   */
+  @tailrec def sourceSymbol(sym: Symbol)(implicit ctx: Context): Symbol =
+    if (!sym.exists)
+      sym


That's completely equivalent, do you want me to change this for style reasons?

smarter · 2017-05-26T12:33:46Z

compiler/src/dotty/tools/dotc/interactive/InteractiveDriver.scala

+
+  private var myCtx: Context = myInitCtx
+
+  def currentCtx: Context = myCtx


Yes, very much so, see usage in DottyLanguageServer

smarter · 2017-05-26T12:34:17Z

compiler/src/dotty/tools/dotc/interactive/InteractiveDriver.scala

+  private val myOpenedTrees = new mutable.LinkedHashMap[URI, List[SourceTree]]
+
+  def openedFiles: Map[URI, SourceFile] = myOpenedFiles
+  def openedTrees: Map[URI, List[SourceTree]] = myOpenedTrees


Yes, see usage in DottyLanguageServer

smarter · 2017-05-26T12:39:57Z

docs/docs/usage/ide-support.md

+
+## Current limitations, to be fixed:
+- Projects should be compiled with sbt before starting the IDE, this is
+  automatically done for you if you run `sbt launchIDE`.


It won't start if the project doesn't compile.

smarter · 2017-05-26T12:41:08Z

language-server/src/dotty/tools/languageserver/DottyLanguageServer.scala

+
+  private[this] var myDrivers: mutable.Map[ProjectConfig, InteractiveDriver] = _
+
+  def drivers: mutable.Map[ProjectConfig, InteractiveDriver] = thisServer.synchronized {


Good point, it shouldn't be.

smarter · 2017-05-26T12:42:36Z

language-server/src/dotty/tools/languageserver/DottyLanguageServer.scala

+
+  def drivers: mutable.Map[ProjectConfig, InteractiveDriver] = thisServer.synchronized {
+    if (myDrivers == null) {
+      assert(rootUri != null, "`drivers` cannot be called before `initialize`")


No, initialize is the first message sent by the client to the server(https://github.com/Microsoft/language-server-protocol/blob/master/protocol.md#initialize), if this message wasn't sent, the serve will not do anything, so this method won't be called.

smarter · 2017-05-26T12:46:43Z

language-server/src/dotty/tools/languageserver/DottyLanguageServer.scala

+    val driver = driverFor(uri)
+
+    val change = params.getContentChanges.get(0)
+    assert(change.getRange == null, "TextDocumentSyncKind.Incremental support is not implemented")


This should never trigger because in initialize we set c.setTextDocumentSync(TextDocumentSyncKind.Full), meaning we don't support TextDocumentSyncKind.Incremental. didChange is a notification, not a request, so the client does not expect a response. Note that this assertion will be caught by LSP4J and so won't bring down the entire language server.

smarter · 2017-05-26T12:48:49Z

language-server/src/dotty/tools/languageserver/DottyLanguageServer.scala

+  }
+
+  override def didChangeConfiguration(params: DidChangeConfigurationParams): Unit =
+    /*thisServer.synchronized*/ {}


A docstring for what? If you're curious what didChangeConfiguration is for, see the spec: https://github.com/Microsoft/language-server-protocol/blob/master/protocol.md#didchangeconfiguration-notification
The commented synchronized is so that if we ever implement this method, we don't forget to make it synchronized

smarter · 2017-05-26T13:13:52Z

language-server/src/dotty/tools/languageserver/Main.scala

+ * This is designed to be started from an editor supporting the Language Server
+ * Protocol, the easiest way to fetch and run this is to use `coursier`:
+ *
+ *   coursier launch $artifact -M dotty.tools.languageserver.Main -- -stdio


I tried it, and it doesn't do any highlighting on this line, so no point in using this syntax.

odersky

Except for minor comments, LGTM

odersky · 2017-05-26T13:30:20Z

compiler/src/dotty/tools/dotc/ast/NavigateAST.scala

+          }))
+
+      if (currentHasPos) {
+        p match {


Should we factor this out into a method forceIfLazy in Positioned?

Maybe, but I think it would be even better if we used something higher-level than dealing with productIterator by hand, that would take care of lazy trees, I'll leave a FIXME in the code.

odersky · 2017-05-26T13:31:15Z

compiler/src/dotty/tools/dotc/ast/NavigateAST.scala

+      val currentHasPos =
+        p.pos.contains(pos) && (!p.pos.isZeroExtent ||
+          (path match {
+            case p1 :: _ => p1.pos.isZeroExtent || !p1.pos.contains(pos)


Would be good to have an explanation of this logic

Indeed, I forgot to clean this up, and trying to document it I realized it's broken: I was trying to get the next node in the tree that the path could go through, but this is actually pretty hard to do. I've given up on that and instead added a skipZeroExtent parameter in NavigateAST#pathTo (that I use in Interactive#pathTo): b3befee

odersky · 2017-05-26T13:46:35Z

compiler/src/dotty/tools/dotc/core/Symbols.scala

@@ -552,6 +553,33 @@ object Symbols {

    type ThisName = TypeName

+    /** If this is a top-level class, and if `-Ykeep-trees` is set, return the tree


Should be -Yretain-trees.

odersky · 2017-05-26T13:47:55Z

compiler/src/dotty/tools/dotc/core/Symbols.scala

@@ -552,6 +553,33 @@ object Symbols {

    type ThisName = TypeName

+    /** If this is a top-level class, and if `-Ykeep-trees` is set, return the tree
+     *  for this class, otherwise null.


Should we use EmptyTree as a sentinel instead? I am not keen on having null as a semantically significant value except when used locally meaning "this cache has not been set".

odersky · 2017-05-26T13:56:40Z

compiler/src/dotty/tools/dotc/core/Symbols.scala

+     */
+    def tree(implicit ctx: Context): tpd.TypeDef = {
+      if (unpickler != null && !denot.isAbsent) {
+        assert(myTree == null)


There's scope to unify the way we do this here with how we store lazy trees in inline functions. Not necessary to do this now, but it would be good to leave a comment telling us to come back to it later.

odersky · 2017-05-26T13:59:43Z

compiler/src/dotty/tools/dotc/core/classfile/ClassfileParser.scala

-            else sym.companionModule.info.member(name).symbol
+            else {
+              var module = sym.companionModule
+              if (module == NoSymbol && sym.isAbsent)


!module.exists is more idiomatic

odersky · 2017-05-26T14:11:58Z

compiler/src/dotty/tools/dotc/interactive/Interactive.scala

+   */
+  @tailrec def sourceSymbol(sym: Symbol)(implicit ctx: Context): Symbol =
+    if (!sym.exists)
+      sym


I think either works fine.

smarter · 2017-05-26T14:55:37Z

java.io.IOException: Cannot run program "code": error=2, No such file or directory

You need code (which is the name of the vscode binary) to be in your PATH. Look at where vscode was installed to find it. I'll update the doc to mention that, but unfortunately I don't know what step-by-step instructions to give to do this on every operating system.

felixmulder · 2017-05-26T14:58:01Z

$ brew cask install visual-studio-code

Is the quick way on macOS

odersky · 2017-05-26T15:19:38Z

@smarter Thanks for the tip. I could launch it now. Looks pretty! I tried it on dotty itself, but so far did not get much to work beyond syntax highlighting. Let's check on Monday together?

smarter · 2017-05-26T15:21:20Z

@odersky Sure, did you get any message in the console you launched it from?

odersky · 2017-05-26T15:25:48Z

@smarter In the console everything looked normal. Here's what I got:

/Users/odersky/workspace/dotty> sbt launchIDE
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0
[warn] Executing in batch mode.
[warn] For better performance, hit [ENTER] to switch to interactive mode, or
[warn] consider launching sbt without any commands, or explicitly passing 'shell'
[info] Loading project definition from /Users/odersky/workspace/dotty/project/project
[info] Loading project definition from /Users/odersky/workspace/dotty/project
[info] Resolving key references (11645 settings) ...
[info] Set current project to dotty (in build file:/Users/odersky/workspace/dotty/)
[warn] Multiple main classes detected. Run 'show discoveredMainClasses' to see the list
[info] Packaging /Users/odersky/workspace/dotty/compiler/target/scala-2.11/dotty-compiler_2.11-0.1.1-bin-SNAPSHOT-nonbootstrapped.jar ...
[info] Done packaging.
[success] Total time: 4 s, completed May 26, 2017 5:17:43 PM
Extension 'lampepfl.dotty' is already installed.
[success] Total time: 0 s, completed May 26, 2017 5:17:43 PM
[warn] Multiple main classes detected. Run 'show discoveredMainClasses' to see the list
[success] Total time: 1 s, completed May 26, 2017 5:17:44 PM

Waiting for source changes... (press enter to interrupt)

smarter · 2017-05-26T16:40:45Z

@odersky: Thanks to @liufengyun I fixed a few issues that could have prevented the IDE from running and updated the branch. But the sbt output you're getting makes me wonder if you're on the correct branch, can you check that you're on branch ide from staging and not ide-support or some other stale branch? Also make sure to get the latest version of the branch after my recent fixes.

FIXME: This is too easy to forget, some sort of Traverser should be used instead of using productIterator by hand.

… package

When typechecking a class from source or when unpickling it from tasty, we normally do not keep a reference to the class tree, but there's a wealth of information only present in the tree that we may wish to use. In an IDE for example, this makes it easy to provide features like "jump to definition". This commit unlocks this potential by adding a `ClassSymbol#tree` method that, when used together with the `-Yretain-trees` flag, returns the tree corresponding to a top-level class. This also replaces the `unpicklers` map in `CompilationUnit` by `ClassSymbol#unpickler`.

When we unpickle a Scala2 module we might not have a companion class to unpickle, when this happen we call markAbsent() on the denotation of the companion class, which means that calls to companionModule will not work. Fixed by falling back to the more robust scalacLinkedClass.

If a tree has no position or a zero-extent position, it should be synthetic. We can preserve this invariant by always setting a zero-extent position for these trees here.

Synthetic trees should get zero-extent positions so that they don't get returned by Interactive#pathTo. This makes it easier to show information related to non-synthetic trees in IDEs.

Previously, the position of class parents was the position of the whole class, which means that NavigateAST#pathTo always returned the class parents when looking up a position in the body of the class. Fixed by using the position of the name of the class instead.

The position of a tree should contain the position of all its children, but that was not the case before this commit for Bind nodes created from Typed nodes.

Before this commit, we only created .tasty files under -YemitTasty, we haven't decided yet if this should be the default, but even if we don't end up doing that, it is still useful to emit empty .tasty files to signal that the corresponding .class file has a tasty section, this is much simpler to check than parsing classfiles to look for a tasty section. Also, fix -YemitTasty tof work with AbstractFile who are not backed by java.io.File, previously this lead to NullPointerException.

As specified in https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.7.16.1, an annotation argument of type boolean, byte, char or short will be represented as a CONSTANT_Integer in the classfile, so when we parse it we get an Int, but before this commit we did not convert this Int to the correct type for the annotation argument, thus creating ill-typed trees.

This way they won't show up in the IDE

This commit adds a set of APIs in dotty.tools.dotc.interactive._ designed for interactive uses of the compiler like in IDEs. This API is just a first draft driven by the requirements of the Dotty Language Server and will evolve based on usage. Usage is roughly as follow: - Create an instance of InteractiveDriver with the compiler flags you need - Call InteractiveDriver#run(uri, code) to typecheck `code` and keep a reference to it via the identifier `uri`. The return value of this method are the errors/warnings/infos generated during the compilation - Use InteractiveDriver#openedTrees(uri) to get all top-level class typechecked trees generated for `uri`, or InteractiveDriver#allTrees to get all top-level trees opened in this driver instance and available on the classpath (unpickled from the TASTY section of classfiles) - Query the trees using one of the many methods in the Interactive object. See the code in language-server/ added in the following commit for a concrete example.

This is an implementation of the Language Server Protocol for Dotty, which allows us to provide rich IDE features in every editor supporting this protocol, a Visual Studio Code extension demonstrating this is part of the next commit. For more information on the LSP, see https://github.com/Microsoft/language-server-protocol Fully supported features: - Typechecking as you type to show compiler errors/warnings - Type information on hover - Go to definition (in the current project) - Find all references Partially working features: - Completion - Renaming - Go to definition in external projects Unimplemented features: - Documentation on hover - Formatting code (requires integrating with scalafmt) - Quick fixes (probably by integrating with scalafix) Current limitations, to be fixed: - Projects should be compiled with sbt before starting the IDE, this is automatically done for you if you run `sbt launchIDE`. - Once the IDE is started, source files that are not opened in the IDE should not be modified in some other editor, the IDE won't pick up these changes. - Not all compiler errors/warnings are displayed, just those occuring during typechecking.

This extension uses the Dotty Language Server to provide IDE features in Visual Studio Code.

This means the dotty build can use the latest unreleased version of the sbt-dotty plugin instead of depending on a published version, in the next commit we'll add IDE support to the sbt-dotty plugin and immediately be able to use them on the dotty build itself.

This adds commands to the sbt-dotty plugin to generate the .dotty-ide.json config file used by the Dotty Language Server and to start Visual Studio Code with everything set up correctly. For end-users, using the IDE is as simple as: 1. Installing vscode (https://code.visualstudio.com) 2. In your project, run `sbt launchIDE`

HerringtonDarkholme · 2017-06-01T10:24:04Z

I encounter a NPE when firing up sbt launcheIDE in https://github.com/lampepfl/dotty-example-project.

Seems https://github.com/lampepfl/dotty/blob/352c09e481cb4893812c5f49959e587842448fe0/language-server/src/dotty/tools/languageserver/DottyLanguageServer.scala#L83 drivers.keys.head throws NPE because drivers is empty. Still investigating what happened here. I will report further info in another issue if I find out why.

smarter · 2017-06-01T10:52:39Z

@HerringtonDarkholme could you post the content of the generated .dotty-ide.json file in the project, as well as the output of the pwd command in a shell in the project directory? What operating system are you on?

smarter · 2017-06-01T10:54:55Z

The exact stacktrace you got would be helpful too

HerringtonDarkholme · 2017-06-02T01:55:04Z

I'm on Mac OSX.

.dotty-ide.json

[ {
  "id" : "root/compile",
  "compilerVersion" : "0.1.2-RC1",
  "compilerArguments" : [ ],
  "sourceDirectories" : [ "/test/dotty-example-project/src/main/scala-0.1", "/test/dotty-example-project/src/main/scala", "/test/dotty-example-project/src/main/java", "/test/dotty-example-project/target/scala-0.1/src_managed/main" ],
  "dependencyClasspath" : [ "/.ivy2/cache/ch.epfl.lamp/scala-library/jars/scala-library-0.1.2-RC1.jar", "/.ivy2/cache/ch.epfl.lamp/dotty-library_0.1/jars/dotty-library_0.1-0.1.2-RC1.jar", "/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.11.11.jar" ],
  "classDirectory" : "/test/dotty-example-project/target/scala-0.1/classes"
} ]

CallStack:

Starting server
Jun 02, 2017 9:18:41 AM org.eclipse.lsp4j.jsonrpc.services.GenericEndpoint notify
INFO: Unsupported notification method: $/setTraceNotification
Jun 02, 2017 9:18:42 AM org.eclipse.lsp4j.jsonrpc.RemoteEndpoint handleNotification

java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
	at org.eclipse.lsp4j.jsonrpc.services.GenericEndpoint.lambda$null$0(GenericEndpoint.java:53)
	at org.eclipse.lsp4j.jsonrpc.services.GenericEndpoint.notify(GenericEndpoint.java:126)
	at org.eclipse.lsp4j.jsonrpc.RemoteEndpoint.handleNotification(RemoteEndpoint.java:165)
	at org.eclipse.lsp4j.jsonrpc.RemoteEndpoint.consume(RemoteEndpoint.java:136)
	at org.eclipse.lsp4j.jsonrpc.json.StreamMessageProducer.handleMessage(StreamMessageProducer.java:149)
	at org.eclipse.lsp4j.jsonrpc.json.StreamMessageProducer.listen(StreamMessageProducer.java:77)
	at org.eclipse.lsp4j.jsonrpc.json.ConcurrentMessageProcessor.run(ConcurrentMessageProcessor.java:84)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.eclipse.lsp4j.jsonrpc.services.GenericEndpoint.lambda$null$0(GenericEndpoint.java:51)
	... 11 more
Caused by: java.lang.NullPointerException
	at scala.collection.mutable.HashTable$$anon$1.next(HashTable.scala:219)
	at scala.collection.mutable.HashTable$$anon$1.next(HashTable.scala:211)
	at scala.collection.mutable.HashMap$$anon$3.next(HashMap.scala:146)
	at scala.collection.IterableLike$class.head(IterableLike.scala:107)
	at scala.collection.AbstractIterable.head(Iterable.scala:54)
	at dotty.tools.languageserver.DottyLanguageServer.driverFor(DottyLanguageServer.scala:83)
	at dotty.tools.languageserver.DottyLanguageServer.didOpen(DottyLanguageServer.scala:143)
	... 16 more

java.lang.NullPointerException
	at scala.collection.mutable.HashTable$$anon$1.next(HashTable.scala:219)
	at scala.collection.mutable.HashTable$$anon$1.next(HashTable.scala:211)
	at scala.collection.mutable.HashMap$$anon$3.next(HashMap.scala:146)
	at scala.collection.IterableLike$class.head(IterableLike.scala:107)
	at scala.collection.AbstractIterable.head(Iterable.scala:54)
	at dotty.tools.languageserver.DottyLanguageServer.driverFor(DottyLanguageServer.scala:83)
	at dotty.tools.languageserver.DottyLanguageServer.hover$$anonfun$1(DottyLanguageServer.scala:286)
	at dotty.tools.languageserver.DottyLanguageServer.liftedTree1$1(DottyLanguageServer.scala:107)
	at dotty.tools.languageserver.DottyLanguageServer.computeAsync$$anonfun$1(DottyLanguageServer.scala:112)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
	at java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:443)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Jun 02, 2017 9:18:42 AM org.eclipse.lsp4j.jsonrpc.RemoteEndpoint lambda$static$0
SEVERE: Internal error: java.lang.NullPointerException
java.util.concurrent.CompletionException: java.lang.NullPointerException
	at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
	at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:604)
	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
	at java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:443)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: java.lang.NullPointerException
	at scala.collection.mutable.HashTable$$anon$1.next(HashTable.scala:219)
	at scala.collection.mutable.HashTable$$anon$1.next(HashTable.scala:211)
	at scala.collection.mutable.HashMap$$anon$3.next(HashMap.scala:146)
	at scala.collection.IterableLike$class.head(IterableLike.scala:107)
	at scala.collection.AbstractIterable.head(Iterable.scala:54)
	at dotty.tools.languageserver.DottyLanguageServer.driverFor(DottyLanguageServer.scala:83)
	at dotty.tools.languageserver.DottyLanguageServer.hover$$anonfun$1(DottyLanguageServer.scala:286)
	at dotty.tools.languageserver.DottyLanguageServer.liftedTree1$1(DottyLanguageServer.scala:107)
	at dotty.tools.languageserver.DottyLanguageServer.computeAsync$$anonfun$1(DottyLanguageServer.scala:112)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
	... 6 more

[Error - 9:18:42 AM] Request textDocument/hover failed.
  Message: Internal error, please look at the server's logs.
  Code: -32603

smarter · 2017-06-02T02:03:40Z

@HerringtonDarkholme My guess is that you have some scala files directly in the directory /test/dotty-example-project. If so, try moving them inside /test/dotty-example-project/src/main/scala (and I'll make sure to fix the IDE support so that this is not needed soon). Otherwise, could you post the output of ls -R /test/dotty-example-project, and the path of the file you tried to open ?

HerringtonDarkholme · 2017-06-02T03:11:46Z

This is my tree -L 2 output.

.
├── LICENSE
├── README.md
├── build.sbt
├── project
│   ├── build.properties
│   ├── plugins.sbt
│   ├── project
│   └── target
├── src
│   └── main
└── target
    ├── resolution-cache
    ├── scala-0.1
    └── streams

wxdong5211 · 2017-06-02T03:17:35Z

me too on windoge 8

HerringtonDarkholme · 2017-06-02T10:20:43Z

I have identified one problem. A for comprehension will catch all Exception. https://github.com/lampepfl/dotty/blob/master/language-server/src/dotty/tools/languageserver/DottyLanguageServer.scala#L65-L69

And alas, InteractiveDriver will initialize Context and thus ScalaSettings where sys.props("user.dir") will be queried.

However, if user has already opened vscode, launchIDE will reuse vscode instance instead of creating new instance in working directory. I personally open vscode from dock where user.dir is a /. Thus https://github.com/lampepfl/dotty/blob/352c09e481cb4893812c5f49959e587842448fe0/compiler/src/dotty/tools/dotc/config/ScalaSettings.scala#L124 throws and swallowed by for.

smarter · 2017-06-02T10:43:00Z

@HerringtonDarkholme Thanks for the diagnostic! Can you try forcing code to run in a new instance using code -n /test/dotty-example-project and see if that helps? I'll make a PR to fix the exception when user.dir is /.

ouchxp · 2017-06-08T01:39:56Z

@smarter code -n /test/dotty-example-project works for me.

HerringtonDarkholme · 2017-06-08T02:28:26Z

Yes, fresh launching makes everything work. ( forget to report back, sorry

smarter · 2017-06-21T20:19:07Z

@wxdong5211 @HerringtonDarkholme Your issues should now be fixed if you use the latest nightly build of dotty (scalaVersion := "0.2.0-bin-20170620-1d05796-NIGHTLY" in your build) and the latest version of sbt-dotty (addSbtPlugin("ch.epfl.lamp" % "sbt-dotty" % "0.1.3") in project/plugins.sbt). Let me know if something is still broken!

HerringtonDarkholme · 2017-06-22T02:09:37Z

It works fine for me! Thank @smarter !

wxdong5211 · 2017-06-22T02:24:24Z

windoge ok too , @smarter 👍

smarter requested review from odersky and felixmulder May 25, 2017 18:29

felixmulder approved these changes May 26, 2017

View reviewed changes

smarter force-pushed the ide branch from e675999 to 55b9ac1 Compare May 26, 2017 13:19

smarter commented May 26, 2017

View reviewed changes

odersky approved these changes May 26, 2017

View reviewed changes

smarter force-pushed the ide branch from 55b9ac1 to 75f517f Compare May 26, 2017 16:37

smarter added 15 commits May 26, 2017 20:00

Compiler#rootContext: do not overwrite initial modes

291656c

Fix navigation with lazy trees

e9f4a07

FIXME: This is too easy to forget, some sort of Traverser should be used instead of using productIterator by hand.

Template denotations should stay valid if their owner is valid

e29195d

Fix exception when calling makePackageObjPrefixExplicit with the root…

30bd0a7

… package

Make "new StoreReporter(null)" work

13f6a91

NavigateAST#pathTo: Add an option to skip over zero-extent nodes

b3befee

Preserve zero-extent positions when setting children positions

2ada3f6

If a tree has no position or a zero-extent position, it should be synthetic. We can preserve this invariant by always setting a zero-extent position for these trees here.

Fix position of implicit conversions, eta-expansion and module vals

f24190d

Synthetic trees should get zero-extent positions so that they don't get returned by Interactive#pathTo. This makes it easier to show information related to non-synthetic trees in IDEs.

Fix position of Bind nodes

bfdf732

The position of a tree should contain the position of all its children, but that was not the case before this commit for Bind nodes created from Typed nodes.

Desugar: Mark label defs as Synthetic

526b9c3

This way they won't show up in the IDE

smarter added 6 commits May 26, 2017 20:00

Visual Studio Code extension for Dotty

0faa6ca

This extension uses the Dotty Language Server to provide IDE features in Visual Studio Code.

Add documentation for the IDE support

dcdbd8e

smarter force-pushed the ide branch from 75f517f to dcdbd8e Compare May 26, 2017 18:00

smarter merged commit 352c09e into scala:master May 26, 2017

MasseGuillaume mentioned this pull request May 30, 2017

autocompletion / autocomplete / code completion scalacenter/scastie#20

Closed

allanrenucci deleted the ide branch December 14, 2017 19:23


		private var myCtx: Context = myInitCtx

		def currentCtx: Context = myCtx


		private[this] var myDrivers: mutable.Map[ProjectConfig, InteractiveDriver] = _

		def drivers: mutable.Map[ProjectConfig, InteractiveDriver] = thisServer.synchronized {

		@@ -552,6 +553,33 @@ object Symbols {

		type ThisName = TypeName

		/** If this is a top-level class, and if `-Ykeep-trees` is set, return the tree

IDE support for Dotty via the Language Server Protocol, including a Visual Studio Code extension #2532

IDE support for Dotty via the Language Server Protocol, including a Visual Studio Code extension #2532

Uh oh!

Conversation

smarter commented May 25, 2017

Quick start

Status

Fully supported features:

Partially working features:

Unimplemented features:

Current limitations, to be fixed:

Future

Uh oh!

smarter commented May 25, 2017

Uh oh!

fommil commented May 26, 2017

Uh oh!

felixmulder left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

smarter left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

odersky left a comment

smarter commented May 26, 2017 •

edited

Loading