Improve decompilation to generate compilable sources

[nicolasstucki]

Currently the decompilation printer scala.tasty.util.ShowSourceCode is the main step in decompilation of class files. It is a glorified TASTY tree pretty printer. It is still quite rough but it prints out a version of the sources that are fully typed (all types are inferred, all implicit are resolved, ...). There are still two kinds of issues to iron out: (i) Resugaring and (ii) leaking internals.

• (i) We need to print some trees that were generated by desugaring back into a source like code. For example, we currently do this for lambdas.
• (ii) The TASTY trees do not match exactly Scala source. We have some internal flags that do not make sense in source code and that should not be printed. Fields are defined in the class and not in the constructor when they should appear in the sources. And many more little details.

The Goal is to have a decompiler that produces back sources that can be recompiled and would generate the same program. We expect to fix this by small incremental steps, supporting more and more features.

Currently, the simplest way to test if some code is decompiled correctly is to create a file Foo.scala (or any of the available tests in tests/pos), write some code in it and run the following scripts from the dotty project:

sbt
;dotc -d output Foo.scala; dotc -decompile -d output -classpath output Foo; dotc -d output2 output/decompiled.scala

If some issue is fixed, regression test must be added by adding a .scala source in tests/pos along with a .decompiled with the same name. For example tests/pos/lambda.scala and tests/pos/lambda.decompiled.

You can test this file by running sbt testFromTasty (or sbt "testFromTasty XYZ" where XYZ is the name of the test), if the test fails a .decompiled.out will be generated besides the .decompiled. You can just rename it if the output of the current run was actually correct.

Note that this file is currently sensitive to spaces at the end of the line (painful and needs to be fixed) and that FromTastyTests append to any existing .decompiled.out which is problematic if you still have one from a previous FromTastyTests. There is also no way to only run a single test under FromTastyTests which is another improvement that must be done.

[nicolasstucki]

Here is a simple one that fails

class Foo
object Foo

generating

class Foo() {}object Foo {}

It just misses a \n between the top level statements.

[nicolasstucki]

```scala class Foo(x: Int) ```

generates

/** Decompiled from output/Foo.class */
class Foo(x: Int) { 
  private[this] val x: Int
}

There is an additional statement created internally to represent the field of the constructor argument. We just need to look at it's flags and filter it out if it is one of those.

[nicolasstucki]

package bar
class Foo {
  protected[bar] def foo(): Int = 0
}

with dotc -decompile ... bar.Foo produces

/** Decompiled from output/bar/Foo.class */
package bar {
  class Foo() { 
    protected def foo(): Int = 0
  }
}

It is missing the bar in protected[bar]. This information can be found in sym.privateWithin of the sym: Symbol of the definition.

[elektronaut0815]

Trying Varargs:

package bar
class Foo {
  def printAll(strings: String*) = strings.map(println)
}

produces

/** Decompiled from output/bar/Foo.class */
package bar {
  class Foo() { 
    def printAll(strings: Seq[String] @scala.annotation.internal.Repeated()): 
      Seq[Unit]
     = 
      strings.map[Unit, Seq[Unit]]((x: Any) => println(x))(
        collection.Seq.canBuildFrom[Unit]
      )
  }
}

Compilation of this works, but annotation shouldn't occur.

[schneist]

class Foo {

  def justdoit (f : Either[Int,String]) : String = {
    f match {
      case Left(i) => i.toString
      case Right(s) => s
    }

  }
}

produces

/** Decompiled from output/Foo.class */
class Foo() {
  def justdoit(f: Either[Int, String]): String =
    {
      f match
        {
          case Left.unapply[Int, String](i @ _): Left[Int, String] =>
            i.toString()
          case Right.unapply[Int, String](s @ _): Right[Int, String] =>
            s: String
        }
    }
}

[elektronaut0815]

Calling varargs method:

package bar
class Foo {
  def printAll(strings: String*) = strings.foreach(println)

  def printDefault() = printAll("One", "Two", "Three")
}

produces

/** Decompiled from output/bar/Foo.class */
package bar {
  class Foo() { 
    def printAll(strings: Seq[String] @scala.annotation.internal.Repeated()): 
      Unit
     = strings.foreach[Unit]((x: Any) => println(x))
    def printDefault(): Unit = 
      this.printAll(["One","Two","Three" : String]: String*)
  }
}

but should be

def printDefault(): Unit =
      this.printAll(Seq("One","Two","Three"): _*)