support in scalac for ignoring hash-bang line in script files

[philwalk]

The current SNAPSHOT reports a "Syntax Error" when compiling a script file with a hash-bang on the first line.
The scalac version tested is:

version:=3.0.0-M3-bin-SNAPSHOT
revision:=73d942cca5936e3f760be327a5a4d6ee9f9c194f
buildTime:=2020-12-16 12:52:04-0700

For example, with the following script file and compile command:

Script file s3new.sc:

#!/usr/bin/env scala3.sh

object S3new {
  //@main def main = printf("YoYo!\n") // does not work, as of dotty-0.27.0-RC1
  def main(args:Array[String]):Unit =
    printf("Yo!\n")
}

$ bin/scalac -color:never -script s3new.sc

+++ dirname bin/scalac
++ cd bin
++ pwd
+ ROOT=/Users/philwalk/workspace/dotty/bin/..
+ /Users/philwalk/workspace/dotty/bin/../bin/common /Users/philwalk/workspace/dotty/bin/../dist/target/pack/bin/scalac -color:never -script s3new.sc
-- [E080] Syntax Error: C:\Users\philwalk\workspace\dotty\s3new.sc:1:0 ---------
1 |#!/usr/bin/env scala3.sh
  |^^^
  |Expected a toplevel definition
1 error found
Error: Errors encountered during compilation

I noticed some references to #! in some source files, but am not yet familiar with the compiler, so it's not clear if this feature is already in the plan:

dotty/tools/dotc/util/SourceFile.scala:23:  private val headerStarts  = List("#!", "::#!")
dotty/tools/dotc/util/SourceFile.scala:27:     *  The header begins with "#!" or "::#!" and ends with a line starting
dotty/scala-backend/src/compiler/scala/tools/cmd/Interpolation.scala:53:    |#!/bin/sh
dotty/scala-backend/src/compiler/scala/tools/nsc/ScriptRunner.scala:18: *    #!/bin/sh
dotty/scala-backend/src/compiler/scala/tools/nsc/ScriptRunner.scala:26: *    ::#!

Without this compiler feature, it becomes unavoidable to make filtered copies of script sources, with compiler error messages referring the the copy rather than the original source file.

[bishabosha]

we could look at using the ScriptSourceFile https://github.com/lampepfl/dotty/blob/33b710a40ca3a154a5f0e7788f91c4ca07085531/compiler/src/dotty/tools/dotc/util/SourceFile.scala#L21

[SethTisue]

I'd love to see this addressed. The same support in Scala 2 has always been really handy.

[SethTisue]

Note that once sbt/sbt#6080 is addressed, it should become possible to use sbt's ScriptMain support as an alternative, but I don't know what the planned timeline there is. (Currently ScriptMain doesn't offer a way to add an sbt plugin, to my knowledge, so you can't add sbt-dotty.)

There's also Ammonite's scripting support; work on Scala 3 support is in progress com-lihaoyi/Ammonite#1135

[philwalk]

Although Scala 2 provides scripting support, the exact above experiment with scalac would also fail in scala 2:

$ scalac jsrc/s3new.sc
\jsrc\s3new.sc:1: error: expected class or object definition
#!/usr/bin/env scala3.sh
^
1 error

In other words, Scala 2 scripting is supported via scala rather than scalac.

It seems that in Scala 3, it's intended that scalac be used to launch scripts, since that is what the -script <scriptName> command line option seems to do. Perhaps to use a single JVM process for both compiling and then running the script (is that how it works?).

It would be convenient if there were support for what python does: the bytecode generated by the compile step is stored in a .pyc or .pyo file, and the compile step is avoided except when the source file is newer than the compiled bytecode.

The optional-compile approach requires a compile step (ideally ignoring hash-bang) with OPTIONAL execution . Then, scripts could be compiled to self-contained .jar files, greatly reducing latency on subsequent script runs, by avoiding the compile step. The re-compile decision could occur in the launcher script.

A natural format would be a jar file with both Class-Path: and Main-Class: properties specified in the jar MANIFEST.MF file. A standard class-path definition could be specified as part of a user execution environment, or by convention (e.g., @$HOME/.scala3classpath). Subsequent runs of the script would be independent of the choice of library imports:

java -jar <scriptFileCompiled.jar>

The runtime classpath would only need to be specified during the compile step, e.g., @ or otherwise.

[SethTisue]

sbt 1.5.0-M1 has direct support for Scala 3, so sbt/sbt#6080 is no longer a blocker for ScriptMain to be supported

I've also opened a sbt ticket on this, sbt/sbt#6274, since work in both repos may be required

[philwalk]

fixed since #11379