Implement @main functions #6898

odersky · 2019-07-20T15:28:42Z

See:

https://github.com/dotty-staging/dotty/blob/add-main-fn/docs/docs/reference/changed-features/main-functions.md

docs/docs/reference/changed-features/main-functions.md

He-Pin · 2019-07-21T07:48:06Z

library/src/scala/main.scala

+
+/** An annotation that designates a main function
+ */
+class main extends scala.annotation.Annotation {}


Are we not making it final?

compiler/src/dotty/tools/dotc/ast/MainProxies.scala

hrhino · 2019-07-21T12:40:53Z

tests/neg/main-functions.scala

+}
+
+object Test2 {
+  @main val x = 2  // does nothing, should this be made an error?


I'd agree that @main val shouldn't be allowed.

I second this. I can't see any good use cases for tagging a val with @main, and thinking about its semantics already stretches the imagination beyond the result being reasonable.

hrhino · 2019-07-21T12:46:08Z

tests/neg/main-functions.scala

+}
+
+class Foo {
+  @main def f = () // does nothing, should this be made an error?


Probably more trouble than it's worth.

Technically this is a correct program that does nothing. I think we should allow empty programs.

soronpo · 2019-07-21T13:59:15Z

Where can I discuss the deprecation of App ?

soronpo · 2019-07-21T14:01:29Z

In what way @main is a "special" feature and not just a private case for annotation macros (which don't exist yet in dotty)?

odersky · 2019-07-21T17:30:39Z

@soronpo If we had annotation macros @main would indeed be expressible with them. But we don't have them, and will probably not get them for 3.0 (maybe later, there are no plans at the moment either way). At the same time we have to do something about the App situation. Current App requires DelayedInit which we don't have either. So we should already promote something else before we go into feature freeze. If @main methods can in the future be expressed with macro annotations so much the better!

soronpo · 2019-07-21T21:01:18Z

docs/docs/reference/changed-features/main-functions.md

+A `@main` annotation on a method turns this method into an executable program.
+Example:
+```scala
+    @main def happyBirthday(age: Int, name: String, others: String*) = {


Should we encourage the use of : Unit here or not?

soronpo · 2019-07-21T21:21:44Z

What is stopping us from annotating an object instead of/in addition to a def?
Currently App is treated differently in some DelayedInit fashion which I understand we wish to get rid of.

object MyApp extends App {
  //do something
}

I think we should also allow @main to work on an object with a def main so App can remain the same, but without extending DelayedInit:

@main object MyApp extends App {
  //do something
}

In my code base I have:

 trait DSLApp extends App { 
  //DSL Library stuff here
}

object DSLUserApp extends DSLApp {
  //User stuff here
}

I would very much like the ability to extend App (or something else), and it does not mean we need to carry the burden of DelayedInit.

jducoeur · 2019-07-21T21:48:14Z

Along the lines of @soronpo's point: the FP community makes pretty heavy use of IOApp, IIRC. So this notion of wanting a specialized main entry point is pretty widespread -- we'd be well-advised to provide some means of supporting that.

odersky · 2019-07-22T07:45:07Z

IOApp is unaffected by this change, you can still use it as is.

The general problem with annotating an object for main is that the main program is the run in the object's initializer code. The JIT tends to be less aggressive in optimizing such code. This is mainly a problem for micro-benchmarks, but that matters, since you get your reputation ruined quickly if people run the same trivial program in Java and Scala and the Scala version is x times slower. DelayedInit gets around that by placing the same code in a method, but that seriously contorts semantics, so we want to get rid of it.

Note that that's not a problem for IOApp, because that one requires you to place your code in the run method.

odersky · 2019-07-22T08:00:46Z

I forgot the more important reason why we do not want to put initalization code in object initializers: It's prone to deadlocks if your program is multi-threaded.

julienrf · 2019-07-22T08:10:52Z

library/src/scala/util/CommandLineParser.scala

+package scala.util
+
+/** A utility object to support command line parsing for @main methods */
+object CommandLineParser {


I understand the goal of simplifying the way to implement application entry points. In practice, most of the users just write a def main(args: Array[String]): Unit method, and manually parse the application arguments. For these cases, this PR definitely improves the developer experience.

That being said, by introducing a way to parse the program arguments in the compiler, we will also establish a “standard” way of doing that. And, I’m a little bit worried about the introduction of the CommandLineParser and FromString abstractions. For now, they are simple but I think we can already anticipate the following requests soon: can we also support optional parameters? Can we support named parameters? How could we customize the error message in case of invalid parameters? Can we print all the error messages instead of stopping at the first one? Should we use Either instead of relying on exceptions to model parsing failures? All these requests are legitimate, in my opinion, but if we try to fulfill them the size of the CommandLineParser and FromString modules will be 10x bigger. Then, the question will be “should we keep that thing in the compiler, or should we move it to the library land (like scopt, or optparse-applicative)?”. We can make the decision of keeping CommandLineParser simple and not honor these requests, “by design”, but then I’m worried about introducing a “standard” way of parsing command-line arguments which does not scale to advanced use cases.

Unfortunately, I don’t have a simple idea of how to decouple @main expansion from CommandLineParser (so that users could plug in the parsing + reporting logic they want). I still think the most flexible way to go is to require users to forward a plain old main method to whatever they want:

def happyBirthday(age: Int, name: String, others: String*): Unit = … def main(args: Array[String]): Unit = { val happyBirthdayParser = … // some code that relies on a library for arguments parsing happyBirthdayParser.run(args)(happyBirthday) }

But I agree that this approach is not beginner-friendly…

I believe we can stay firm and keep CommandLineParser simple. If one wants something else, one can perfectly well define a traditional main method, or define the entry point like this:

@main def f(args: String*) = ...

liufengyun · 2019-07-22T08:26:56Z

I find the following is more natural, no semantic magic with names:

The code

    @main class HappyBirthday(age: Int, name: String, others: String*) {
        // code
    }

expands to

    class HappyBirthday(age: Int, name: String, others: String*) {
	// code
    }

    object HappyBirthday {
      import scala.util.{CommndLineParser => CLP}
      def main(args: Array[String]): Unit = {
	try
	  new HappyBirthday(
	      CLP.parseArgument[Int](args, 0),
	      CLP.parseArgument[String](args, 1),
	      CLP.parseRemainingArguments[String](args, 2))
	catch {
	  case error: CLP.ParseError => CLP.showError(error)
	}
      }
    }

liufengyun · 2019-07-22T09:20:51Z

I don’t see how benchmarking is an issue, as usage of JMH will requires the code to be in a method.

soronpo · 2019-07-22T13:43:00Z

@odersky is supporting both forms (@main object ..., @main def ...) out of the question? The latter is used in most cases and the first to ease migration and allow the example I've given above.
Or alternatively if we can do @main class MyApp extends (Array[String] => Unit) {}, that is fine too I believe.

odersky · 2019-07-22T14:19:57Z

@odersky is supporting both forms (@main object ..., @main def ...) out of the question?

For me, yes. We don't need more than one convenient way to do main programs. @main def has the advantage that it's what ammonite does, so we close the gap between scripting and normal programs.

anatoliykmetyuk

Looks fine to me, except the points specified below. Another thing: doesn't work with SBT.

From Dotty repo, on this PR branch, do dotty-bootstrapped/publishLocal. Then:

sbt new lampepfl/dotty.g8 --name mainmethods
cd mainmethods

In buld.sbt, set:

val dottyVersion = "0.18.0-bin-SNAPSHOT"

Then in Main.scala, write:

object Main {
  // def main(args: Array[String]): Unit = println("foo")

  @main def f(): Unit = {
    println("Hello world!")
    println(msg)
  }

  def msg = "I was compiled by dotty :)"

}

Then do sbt run:

[error] java.lang.RuntimeException: No main class detected.
[error] 	at scala.sys.package$.error(package.scala:26)
[error] (Compile / bgRun) No main class detected.
[error] Total time: 11 s, completed Jul 26, 2019 2:27:44 PM

However, if you compile with dotc instead, it works (I tested it from SBT console):

dotc ../ecosystem/mainmethods/src/main/scala/Main.scala
dotr f

(change the argument to dotc to the path to Main.scala on your machine).

Now in Main.scala, uncomment def main(args: Array[String]): Unit and run dotc again. Then from Dotty repo in bash, do:

tsf-428-wpa-0-238:dotty anatolii$ javap Main$ f
Compiled from "Main.scala"
public final class Main$ implements scala.Serializable {
  public static final Main$ MODULE$;
  public static {};
  public void main(java.lang.String[]);
  public void f();
  public java.lang.String msg();
}
Compiled from "Main.scala"
public final class f {
  public f();
  public static void main(java.lang.String[]);
}

I believe sbt may not be prepared to the fact that main methods may be static and/or come from classes and not objects (i.e. classes that end with $). IMO it is best to keep the new main methods compliant to the way the main methods are usually expressed in Scala.

anatoliykmetyuk · 2019-07-26T11:15:23Z

docs/docs/reference/changed-features/main-functions.md

+  }
+```
+**Note**: The `<static>` modifier above expresses that the `main` method is generated
+as a static method of class `happyBirthDay`. It is not available for user programs in Scala. Regular "static" members are generated in Scala using objects instead.


Why do we make this design decision here? Looks like a limitation to me. E.g. I may want to write a library of utility functions that may depend on each other:

@main def disableFile(path: Path): Unit = // Change the `path`'s extension from `scala` to `disabled` @main def enableFile(path: Path): Unit = // Change the `path`'s extension from `disabled` to `scala` @main def filterFolder(folder: Path, fileNamePattern: Regex): Unit = { // Disable all the files except those the names of which match the given regex pattern val allFiles: List[Path] = // get all the files present in the folder allFiles.foreach(enableFile) // Make sure we start with a clean folder, unaffected by previous runs of this or sister programs allFiles.filterNot(file => fileNamePattern.matches(file.name)).foreach(disableFile) }

It is not available for user programs in Scala

Also, it doesn't seem that this limitation is true:

object Main { def main(args: Array[String]): Unit = { f(); println("foo") } @main def f(): Unit = { println("Hello world!") println(msg) } @main def g(): Unit = f() def msg = "I was compiled by dotty :)" }

Compiles and runs fine.

Doesn't that mean that the synthetic method main can't be called, rather than that the @main method written in source can't be called?

anatoliykmetyuk · 2019-07-26T11:16:59Z

docs/docs/reference/changed-features/main-functions.md

+the required types. If a check fails, the program is terminated with an error message.
+Examples:
+```
+> scala happyBirthday 22


What is the exact lifecycle of such a program here? What's happyBirthday? Do we need to compile it with dotc and run scala from the folder in which the class files end up? Can we do scala happyBirthday.scala 22 – that is without the necessity to compile the source? I think we need to elaborate on this in the docs.

anatoliykmetyuk · 2019-07-26T11:18:07Z

library/src/scala/main.scala

+
+/** An annotation that designates a main function
+ */
+class main extends scala.annotation.Annotation {}


Are we not making it final?

library/src/scala/util/FromString.scala

tests/neg-custom-args/fatal-warnings/main-functions-nameclash.scala

anatoliykmetyuk · 2019-07-26T11:35:39Z

tests/neg/main-functions.scala

+}
+
+object Test2 {
+  @main val x = 2  // does nothing, should this be made an error?


I second this. I can't see any good use cases for tagging a val with @main, and thinking about its semantics already stretches the imagination beyond the result being reasonable.

anatoliykmetyuk · 2019-07-26T11:36:32Z

tests/neg/main-functions.scala

+}
+
+class Foo {
+  @main def f = () // does nothing, should this be made an error?


Technically this is a correct program that does nothing. I think we should allow empty programs.

anatoliykmetyuk · 2019-07-26T11:39:11Z

tests/run/main-functions.scala

+  println("Hello, world!")
+
+object A {
+  @main def foo(x: Int, y: String, zs: Float*) =


Is this code executed anywhere?

library/src/scala/util/CommandLineParser.scala

compiler/src/dotty/tools/dotc/ast/MainProxies.scala

smarter · 2019-07-26T15:09:13Z

I believe sbt may not be prepared to the fact that main methods may be static and/or come from classes and not objects (i.e. classes that end with $).

We control main class detection, not sbt: https://github.com/lampepfl/dotty/blob/232b3fa9e17a771e55088299d5e461ee25aed9e5/compiler/src/dotty/tools/dotc/sbt/ExtractAPI.scala#L233-L235

odersky · 2019-07-26T15:47:45Z

Thanks for the hint @smarter! I tried a fix. @anatoliykmetyuk Can you test whether it works now with sbt?

allanrenucci · 2019-07-26T16:26:33Z

compiler/src/dotty/tools/dotc/sbt/ExtractAPI.scala

@@ -230,7 +230,7 @@ private class ExtractAPICollector(implicit val ctx: Context) extends ThunkHolder

    allNonLocalClassesInSrc += cl

-    if (sym.isStatic && defType == DefinitionType.Module && ctx.platform.hasMainMethod(sym)) {
+    if (sym.isStatic && ctx.platform.hasMainMethod(sym)) {


I think you need to drop sym.isStatic to allow classes to have main method.

smarter · 2019-07-26T17:48:12Z

We should add a scripted sbt test to verify the main class detection works: https://github.com/lampepfl/dotty/tree/master/sbt-dotty/sbt-test/sbt-dotty

anatoliykmetyuk · 2019-07-26T18:24:04Z

As of 8b2737d, SBT detects the @main methods

Check applicability of annotations only once they are completed

If we make this method take an implicit function rather than a normal one we get a build error with trace: ``` [error] ## Exception when compiling 9 sources to /Users/odersky/workspace/dotty/sbt-bridge/src/target/classes [error] Type scala.ImplicitFunction1 not present [error] sun.reflect.generics.factory.CoreReflectionFactory.makeNamedType(CoreReflectionFactory.java:117) [error] sun.reflect.generics.visitor.Reifier.visitClassTypeSignature(Reifier.java:125) [error] sun.reflect.generics.tree.ClassTypeSignature.accept(ClassTypeSignature.java:49) [error] sun.reflect.generics.repository.ConstructorRepository.getParameterTypes(ConstructorRepository.java:94) [error] java.lang.reflect.Executable.getGenericParameterTypes(Executable.java:283) [error] java.lang.reflect.Method.getGenericParameterTypes(Method.java:283) [error] sbt.internal.inc.ClassToAPI$.parameterTypes(ClassToAPI.scala:566) [error] sbt.internal.inc.ClassToAPI$.methodToDef(ClassToAPI.scala:318) [error] sbt.internal.inc.ClassToAPI$.$anonfun$structure$1(ClassToAPI.scala:182) [error] sbt.internal.inc.ClassToAPI$.$anonfun$mergeMap$1(ClassToAPI.scala:400) [error] scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:240) [error] scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:32) [error] scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:29) [error] scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:37) [error] scala.collection.TraversableLike.flatMap(TraversableLike.scala:240) [error] scala.collection.TraversableLike.flatMap$(TraversableLike.scala:237) [error] scala.collection.AbstractTraversable.flatMap(Traversable.scala:104) [error] sbt.internal.inc.ClassToAPI$.merge(ClassToAPI.scala:411) [error] sbt.internal.inc.ClassToAPI$.mergeMap(ClassToAPI.scala:400) [error] sbt.internal.inc.ClassToAPI$.structure(ClassToAPI.scala:182) [error] sbt.internal.inc.ClassToAPI$.x$2$lzycompute$1(ClassToAPI.scala:133) [error] sbt.internal.inc.ClassToAPI$.x$2$1(ClassToAPI.scala:133) [error] sbt.internal.inc.ClassToAPI$.instance$lzycompute$1(ClassToAPI.scala:133) [error] sbt.internal.inc.ClassToAPI$.instance$1(ClassToAPI.scala:133) [error] sbt.internal.inc.ClassToAPI$.$anonfun$toDefinitions0$1(ClassToAPI.scala:140) [error] xsbti.api.SafeLazyProxy$$anon$1.get(SafeLazyProxy.scala:26) [error] xsbti.api.SafeLazy$Impl.get(SafeLazy.java:58) ```

odersky · 2019-07-29T20:20:02Z

@anatoliykmetyuk I think everything we agreed on is done. Merge?

deanwampler · 2020-08-22T14:12:13Z

Has any thought been given to user-pluggable help messages for bad user input? The output messages are quite minimalistic, which is fine as a default, but not very friendly to the user. A minimal addition would be the ability to define a pluggable help message (or better, a method) that is called to create the equivalent of --help output, which is printed after printing the error message. Say, for example:

@help def helpCLI(same_arg_list?): String = """
   stuff
"""

I think something like this will be mandatory if def main() is eventually deprecated.

julienrf · 2020-08-22T17:02:38Z

The system will be extensible to some degree (there is a discussion on cotributors.scala-lang.org), but if you need more power you should fallback to a user-land solution such as scopt or decline.

nafg suggested changes Jul 21, 2019

View reviewed changes

docs/docs/reference/changed-features/main-functions.md Outdated Show resolved Hide resolved

He-Pin reviewed Jul 21, 2019

View reviewed changes

hrhino reviewed Jul 21, 2019

View reviewed changes

soronpo reviewed Jul 21, 2019

View reviewed changes

julienrf reviewed Jul 22, 2019

View reviewed changes

odersky requested a review from anatoliykmetyuk July 25, 2019 16:47

odersky assigned anatoliykmetyuk Jul 25, 2019

anatoliykmetyuk suggested changes Jul 26, 2019

View reviewed changes

odersky force-pushed the add-main-fn branch from a3fb089 to b935782 Compare July 26, 2019 15:06

allanrenucci reviewed Jul 26, 2019

View reviewed changes

odersky force-pushed the add-main-fn branch from 8de209b to 8b2737d Compare July 26, 2019 17:18

odersky force-pushed the add-main-fn branch 3 times, most recently from 18c2061 to 5cdf2d6 Compare July 27, 2019 11:05

odersky mentioned this pull request Jul 27, 2019

Using an implicit function type in Reporter breaks the build #6946

Closed

odersky added 13 commits July 29, 2019 18:54

Implement @main functions

a5df0e7

Add docs

fce621c

Make main proxies final and docs fixes

97c1cd2

Link main-functions.html in sidebar

e1b10c8

Fix typo

ef16db7

Improve clash error message

a2b6023

Allow static class methods as main methods

c53f465

Check @main annotations for applicability

eae2382

Check applicability of annotations only once they are completed

Add test

ffea9f4

Use given as for FromString instances

4df2db1

Fix annotation applicability checking

622e8c3

Disallow @main methods with implicit parameters

eb374a1

odersky force-pushed the add-main-fn branch from 16b3efe to eb374a1 Compare July 29, 2019 16:55

anatoliykmetyuk approved these changes Jul 30, 2019

View reviewed changes

anatoliykmetyuk merged commit b3826d0 into scala:master Jul 30, 2019

anatoliykmetyuk deleted the add-main-fn branch July 30, 2019 08:05

This was referenced Jul 30, 2019

Improve @main methods docs lampepfl/dotty-knowledge#16

Open

Add tests for @main functions integration with SBT lampepfl/dotty-knowledge#17

Closed

nicolasstucki mentioned this pull request Jul 31, 2019

Nightly CI build is failing test_sbt #6972

Closed

liufengyun mentioned this pull request Aug 21, 2019

dotr should exec scripts #4747

Closed

anatoliykmetyuk added this to the 0.18 Tech Preview milestone Aug 28, 2019

Implement @main functions #6898

Implement @main functions #6898

Uh oh!

Conversation

odersky commented Jul 20, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

soronpo commented Jul 21, 2019

Uh oh!

soronpo commented Jul 21, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

odersky commented Jul 21, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

soronpo commented Jul 21, 2019

Uh oh!

jducoeur commented Jul 21, 2019

Uh oh!

odersky commented Jul 22, 2019

Uh oh!

odersky commented Jul 22, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

liufengyun commented Jul 22, 2019

Uh oh!

liufengyun commented Jul 22, 2019

Uh oh!

soronpo commented Jul 22, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

odersky commented Jul 22, 2019

Uh oh!

anatoliykmetyuk left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

smarter commented Jul 26, 2019

Uh oh!

odersky commented Jul 26, 2019

Uh oh!

odersky commented Jul 20, 2019 •

edited

Loading

soronpo commented Jul 21, 2019 •

edited

Loading

soronpo commented Jul 22, 2019 •

edited

Loading

smarter commented Jul 26, 2019 •

edited

Loading