One click install for Scala

[sjrd]

Update the download page with the brand new one-click install developed by @alexarchambault for the Scala Center, and add a blog post to give it visibility :)

[alexarchambault]

That looks fine to me. Is there an easy way to preview the result?

[sjrd]

Here is a rendering of the download page for Windows:

[julienrf]

Should we add some details about how we manage specific versions of scala and scalac, and how to update the cs tool itself?

[sjrd]

Done as well.

[lrytz]

For me it's the first time I looked at the cs native launcher a little bit closer, also the first time I looked at cs setup, cs install and cs java. Since the download page also targets beginners, maybe mostly beginners, that's probably a good starting point for a review 🙂

First of all, cs looks really great, I'm looking forward to this change!

I worry about how "magic" it all seems when just looking at the scala-lang webiste, and coursier's CLI install page that is linked. The "one click" / "one command" to set up everything is the perfect starting point, but after doing that, people will wonder

• where does it install all the stuff?
• how does it modify my environment?
• what java (edition, version) does it install?
• ...

These things matter, even to begginers - a Scala beginner is not necessarily programmer beginner. Even more, cs is not only for beginners, so experts should feel confident to start using it!

Of course we cannot explain these things on the download page, but it should be clear that cs setup is not a one-shot script that does some stuff to your system and then leaves you alone. We should explain what cs is and roughly how it works. Maybe also improve the documentation on the coursier website, because if scala-lang->download links to https://get-coursier.io/docs/cli-overview.html#install-native-launcher, then that page really becomes an entry point into the coursier website.

Things that are unclear to me after browsing through the coursier website

• How to install a specific version, and switch between versions? I think this is important at least for scala, not everyone wants 2.13 right now for example.
• What is latest.release? Does cs always automatically upgrade to the latest release? Is that a good idea?
• cs can install cs, right? Can i delete the native launcher after cs setup? What if I used brew install?

Finally, for me the download of the native launcher was extremely slow, i tried on mac and windows (it wasn't due to my connection).

[sjrd]

Reverted to using the old content with sbt instead of VS Code, until we have a proper getting started guide with VS Code.

[sjrd]

@alexarchambault I guess @lrytz's comment is best addressed by you?

[sjrd]

Here is a new rendering:

[SethTisue]

I worry a bit if this stuff — which is still rather new — has been tested enough, on a wide enough variety of setups, to be safe enough to recommend to newcomers.

(And newcomers are especially likely to just bail and disappear, rather than let us know that something isn't working for them.)

Should I worry, or relax?

[sjrd]

At least for Windows, it works better than any of the individual tools' install mechanisms. It's really just double-clicking on the exe, and it installs everything, and then everything just works.

It may not have been tested by a large number of people, true. We specifically made a call for testers for Windows a week or two ago. For other systems, I don't know.

[julienrf]

Here is a screenshot of what it looks like from a Linux machine:

[tpolecat]

I agree with comments from @lrytz. I want to know what version of what is installed where … when I type java is it the same VM that's used when I call scala? How do I switch versions? Will all my tooling also switch versions with me (Ammonite for example, which is tied to a specific Scala version)? And so on. This seems great but it will prompt a lot of questions.

[sjrd]

@julienrf Hum that's not normal. Those are the generic instructions that it falls back on if it can't tell the OS. It should display something like this:

$ curl -Lo cs https://git.io/coursier-cli-linux && chmod +x cs && ./cs setup

Did you get any error in the browser's console?

[julienrf]

No, but it seems that the os detection is not very good. It is based on window.navigator.appVersion. On my OS its value is “5.0 (X11)”, which is then identified to be “unix”, that’s why I get these instructions rather than the “linux” instructions.

Maybe that function is more robust? https://stackoverflow.com/questions/38241480/detect-macos-ios-windows-android-and-linux-os-with-js With this one, my system is correctly identified as Linux.

[julienrf]

@alexarchambault Would it be possible to update the documentation of coursier to answer all the questions raised by Lukas and Rob? Eventually, we need to copy parts of this documentation in the blog post and in the getting started page.

[exoego]

I think the link to JDK compatibility, that was recently added in #1123, should be preserved, so new comers do not get trouble when they use latest JDK and older Scala combination.

[alexarchambault]

Thanks for your comments everyone. I'd like to give some more details about the setup / install / java commands before answering @lrytz's and @tpolecat's points.

About cs setup

The cs setup command is just some "repackaging" of features of the cs java and cs install commands. One doesn't have to run cs setup, it's just a convenience - for beginners in particular. By relying on commands that more advanced users can benefit from, cs setup should be kept up-to-date and more maintained in the future.

The cs java command (managing JVMs) and cs install (managing applications) are quite independent of each other. One can use one and not the other. One can keep managing JVMs another way, while using cs install. Or one can use JVMs from cs java while not using cs install. It's also possible to use these two commands while not installing anything globally - not changing profile files / user environment variables, and keeping everything in the coursier cache, see below.

About cs java

cs java manages JVMs. For example, running cs java --jvm 11 -version will automatically download the latest AdoptOpenJDK 11 (in the coursier cache), unpack it (in the managed JVM directory), and run it with -version:

$ cs java --jvm 11 -version
openjdk version "11.0.6" 2020-01-14
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.6+10)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.6+10, mixed mode)

It uses the index of jabba to know where to download JVM archives, and assumes AdoptOpenJDK if only a version is passed.

cs java-home prints the Java home of a JVM, like

$ cs java-home --jvm 9
~/Library/Caches/Coursier/jvm/adopt@1.9.0-0/Contents/Home

One can update profile files (Linux / macOS) or user environment variables (Windows) for a specific JVM with --setup, like

$ cs java --jvm openjdk:1.14 --setup

(this updates JAVA_HOME and PATH)

cs setup does basically that, for the default JVM (AdoptOpenJDK 8).

But you don't have to update these environment variables globally. For example, one can just get a Java home with

$ cs java-home --jvm graalvm:20
~/Library/Caches/Coursier/jvm/graalvm@20.0.0/Contents/Home

or set environment variables only for the current session with

$ eval "$(cs java --env --jvm 11)"
$ java -version
openjdk version "11.0.6" 2020-01-14
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.6+10)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.6+10, mixed mode)

These two commands only write things in the coursier cache, and the managed JVM directory, which is a cache too (both of these directories live under ~/.cache/coursier on Linux, ~/Library/Caches/Coursier on macOS).

About cs install

The cs install command creates launchers for applications, in the installation directory (~/.local/share/coursier/bin on Linux). One can pass --install-dir to cs install and cs setup to install launchers in another directory.

One can run cs install --setup to update profile files (Linux / macOS) or user environment variables (Windows) (this adds the installation directory to PATH). cs setup does that too.

But one can also rely on applications… while not installing them, with cs launch, like

$ cs launch scala:2.12.10
Welcome to Scala 2.12.10 (OpenJDK 64-Bit Server VM, Java 11.0.6).
Type in expressions for evaluation. Or try :help.

scala> :q

or

$ cs launch ammonite:2.0.4 --scala 2.12.8
cs launch ammonite:2.0.4 --scala 2.12.8
Loading...
Welcome to the Ammonite Repl 2.0.4 (Scala 2.12.8 Java 1.8.0_121)
@

Lastly, one can run applications with specific JVMs with

$ cs launch --jvm 11 --scala 2.12.9 ammonite:2.0.4
…

(this particular command suffers from a regression as of writing this, should be fixed soon).

[lrytz]

Thank you @alexarchambault for the summary! I did manage do get most of what you descibe by reading/skimming through the coursier website. For someone that's completely new to coursier, or coming from the scala download page, this kind of summary should be the first thing that's shown.

A few specific questions:

• Is it a good idea that cs install scala doesn't support picking a version, and that it auto-upgrades? My intuition is that this should be more manual (like what I'm used to from homebrew).
• After running cs setup for the first time, cs installs itself right? Should we recommend to delete the download?
• Running just cs on mac does nothing - should it display --help? On windows, just running cs re-runs cs setup. This should probably not be the default once you have it installed on the command line. So maybe it would be a good idea if the one-click-installer was a separate executable that does cs setup when double-clicked, but then the cs version installed by cs doesn't do that, but shows --help?
• The native launcher downlaods are really slow - can this be improved?

[alexarchambault]

Is it a good idea that cs install scala doesn't support picking a version, and that it auto-upgrades?

You can pick a version with cs install, like

$ cs install scala:2.12.10

For now, running cs update will bump it to the latest version though. That needs to be fixed. (Is is what you meant by "auto-upgrades"?)

After running cs setup for the first time, cs installs itself right? Should we recommend to delete the download?

We can, yes, and maybe should actually.

Running just cs on mac does nothing - should it display --help?

Good point, I guess it should, that needs to be fixed.

On windows, just running cs re-runs cs setup. This should probably not be the default once you have it installed on the command line. So maybe it would be a good idea if the one-click-installer was a separate executable that does cs setup when double-clicked, but then the cs version installed by cs doesn't do that, but shows --help?

That's a good point. It's the same launcher for now only because it's easier (and faster) to generate and upload one launcher rather than two, from the coursier CI (via this script).

The native launcher downlaods are really slow - can this be improved?

Maybe we should use curl rather than bitsadmin in the instructions to download it. bitsadmin is supposed to download things with a low priority somehow. And recent versions of Windows are supposed to have curl pre-installed.

[alexarchambault]

To answer @tpolecat's points:

I want to know what version of what is installed where …

Currently, this is kept in some metadata files written alongside launchers (the launchers are actually zip files, these metadata are under META-INF/coursier/ in them, even for the native applications). Maybe we could add a command to list installed applications with their versions.

when I type java is it the same VM that's used when I call scala?

Yes. The installed application launchers, and the installed JVMs, are quite independent from each other. They aren't even aware of each other. The app launchers will pick whatever version of java is installed on the current machine (via PATH, JAVA_HOME, …) They will pick a JVM installed by cs setup or cs java only if it's already setup via PATH or JAVA_HOME.

How do I switch versions?

For the scala launcher, it may be possible to offer to change the scala version by relying on the scripts from @dwijnand's scala-runners. For now, the scala launcher starts the version that was choosen upon installation, which is the latest by default.

But it's possible to run specific versions via cs launch, like

$ cs launch scala:2.12.0
…
$ cs launch scala:2.12.0 -- …options…
…

Will all my tooling also switch versions with me (Ammonite for example, which is tied to a specific Scala version)?

No. The launchers are independent of each others. In more detail, they're tiny Java applications, with a list of URLs in a resource. Upon launch, they read that URL list, find the corresponding JARs in the coursier cache, and start that. So you can have scala start scala 2.12.10 (if you did cs install scala:2.12.10) and Ammonite use scala 2.13.1. Both pick their JARs in the coursier cache, independently of each other, just like sbt or mill do for library dependencies.

[tpolecat]

@alexarchambault 👍 thanks for the explanations. As @lrytz suggests, I think it would be helpful to display this information prominently, and/or make it available via --help for specific commands. Because this is a new way to install and manage things it's important for people to understand what exactly is going on.

[alexarchambault]

I think it would be helpful to display this information prominently, and/or make it available via --help for specific commands.

Agreed… I'll try to fix that.

[sjrd]

Thanks! Was this a Windows-only issue? What would be the cost of making sure this doesn't happen anymore? The link for the download automatically point to the latest release. Once we make it the default, we cannot afford the released installer to be broken like that.

[alexarchambault]

Was this a Windows-only issue?

Yes, the code that crashed is only run on Windows (it's related to encoding some powershell code in base64 prior to running it…)

What would be the cost of making sure this doesn't happen anymore?

Right now, the native launchers are only generated upon release, as these take time to generate. I'm thinking about running some tests before pushing them, and / or maybe run those on a daily basis, so that we can catch regressions not too long after they occurred.

[alexarchambault]

About

On windows, just running cs re-runs cs setup. This should probably not be the default once you have it installed on the command line. So maybe it would be a good idea if the one-click-installer was a separate executable that does cs setup when double-clicked, but then the cs version installed by cs doesn't do that, but shows --help?

this should be addressed in the upcoming release. It'll show the help message when run with no arguments, rather than not printing anything.

On Windows, the installed cs will also show the help message when run with no arguments. Some environment variables are set right before the installed native launcher starts, that allows it to detect if it's install-ed (show the help) or not (assume the user double-clicked on it, and run setup).

[alexarchambault]

Just a heads up, to tell that I think most issues around the sbt launcher were addressed.

• It now relies on sbt/launcher. It's much closer to the standard sbt launcher now.
• It now supports options like --supershell=…. This relies on some custom option parsing logic, via coursier/sbt-runner, that runs before the main class of sbt/launcher. This logic doesn't come straight from sbt/sbt-launcher-package, but it can be evolved alongside it. coursier/sbt-runner has been discussed here.
• It supports reading java options from .jvmopts.

You need to install / update sbt with the latest cs for all of that to work, like

$ cs update cs
$ cs
Coursier 2.0.0-RC6-21
…
$ cs update sbt
Updated sbt

(If the last step found no new update, do cs uninstall sbt && cs install sbt to re-generate the sbt launcher.)

[alexarchambault]

We've been gathering feedback about the Windows launcher here, and introduced the setup command to a wider audience here.

[sjrd]

Following all the discussions and a suggestion of @julienrf, I have removed the changes to the download page, and only kept the blog post, so that we can move on by announcing the existence of the one-click install, encourage some people to test it more widely, but keep from recommending it by default until it's been tested more.

I have kept the changes to the download page in a separate branch for safekeeping: https://github.com/sjrd/scala-lang/tree/one-click-install-by-default

[lrytz]

👍

[SethTisue]

Perhaps the blog post could open with some kind of indication of what lies ahead. And/or some kind of call-to-action for the community; what do we want people to do with this information?

[julienrf]

Perhaps the blog post could open with some kind of indication of what lies ahead. And/or some kind of call-to-action for the community; what do we want people to do with this information?

Actually, I think the first paragraph already does that.

[SethTisue]

scala_lang tweet?

[julienrf]

Yes, good point. Fixed :) https://twitter.com/scala_lang/status/1278249350523748352

[martijnhoekstra]

Following all the discussions and a suggestion of @julienrf, I have removed the changes to the download page, and only kept the blog post, so that we can move on by announcing the existence of the one-click install, encourage some people to test it more widely, but keep from recommending it by default until it's been tested more.

I have kept the changes to the download page in a separate branch for safekeeping: https://github.com/sjrd/scala-lang/tree/one-click-install-by-default

This is the future calling. Maybe the time is now.

[bishabosha]

https://alexarchambault.github.io/posts/2020-09-21-cs-setup.html, very good information about the usage