Extreme long type definition causes a java.lang.StackOverflowError

[devlaam]

Compiler version

3.3.3 and 3.4.1

Minimized code

type Symbol = 
    "0" | "0" | "circle" | "100" | "100" | "circle-open" | "200" |
    "200" | "circle-dot" | "300" | "300" | "circle-open-dot" | "1" | "1" | "square" | "101" | "101" | "square-open" |
    "201" | "201" | "square-dot" | "301" | "301" | "square-open-dot" | "2" | "2" | "diamond" | "102" | "102" |
    "diamond-open" | "202" | "202" | "diamond-dot" | "302" | "302" | "diamond-open-dot" | "3" | "3" | "cross" | "103" |
    "103" | "cross-open" | "203" | "203" | "cross-dot" | "303" | "303" | "cross-open-dot" | "4" | "4" | "x" | "104" |
    "104" | "x-open" | "204" | "204" | "x-dot" | "304" | "304" | "x-open-dot" | "5" | "5" | "triangle-up" | "105" |
    "105" | "triangle-up-open" | "205" | "205" | "triangle-up-dot" | "305" | "305" | "triangle-up-open-dot" | "6" |
    "6" | "triangle-down" | "106" | "106" | "triangle-down-open" | "206" | "206" | "triangle-down-dot" | "306" | "306" |
    "triangle-down-open-dot" | "7" | "7" | "triangle-left" | "107" | "107" | "triangle-left-open" | "207" | "207" |
    "triangle-left-dot" | "307" | "307" | "triangle-left-open-dot" | "8" | "8" | "triangle-right" | "108" | "108" |
    "triangle-right-open" | "208" | "208" | "triangle-right-dot" | "308" | "308" | "triangle-right-open-dot" | "9" |
    "9" | "triangle-ne" | "109" | "109" | "triangle-ne-open" | "209" | "209" | "triangle-ne-dot" | "309" | "309" |
    "triangle-ne-open-dot" | "10" | "10" | "triangle-se" | "110" | "110" | "triangle-se-open" | "210" | "210" |
    "triangle-se-dot" | "310" | "310" | "triangle-se-open-dot" | "11" | "11" | "triangle-sw" | "111" | "111" |
    "triangle-sw-open" | "211" | "211" | "triangle-sw-dot" | "311" | "311" | "triangle-sw-open-dot" | "12" | "12" |
    "triangle-nw" | "112" | "112" | "triangle-nw-open" | "212" | "212" | "triangle-nw-dot" | "312" | "312" |
    "triangle-nw-open-dot" | "13" | "13" | "pentagon" | "113" | "113" | "pentagon-open" | "213" | "213" |
    "pentagon-dot" | "313" | "313" | "pentagon-open-dot" | "14" | "14" | "hexagon" | "114" | "114" | "hexagon-open" |
    "214" | "214" | "hexagon-dot" | "314" | "314" | "hexagon-open-dot" | "15" | "15" | "hexagon2" | "115" | "115" |
    "hexagon2-open" | "215" | "215" | "hexagon2-dot" | "315" | "315" | "hexagon2-open-dot" | "16" | "16" | "octagon" |
    "116" | "116" | "octagon-open" | "216" | "216" | "octagon-dot" | "316" | "316" | "octagon-open-dot" | "17" | "17" |
    "star" | "117" | "117" | "star-open" | "217" | "217" | "star-dot" | "317" | "317" | "star-open-dot" | "18" |
    "18" | "hexagram" | "118" | "118" | "hexagram-open" | "218" | "218" | "hexagram-dot" | "318" | "318" |
    "hexagram-open-dot" | "19" | "19" | "star-triangle-up" | "119" | "119" | "star-triangle-up-open" | "219" | "219" |
    "star-triangle-up-dot" | "319" | "319" | "star-triangle-up-open-dot" | "20" | "20" | "star-triangle-down" | "120" |
    "120" | "star-triangle-down-open" | "220" | "220" | "star-triangle-down-dot" | "320" | "320" |
    "star-triangle-down-open-dot" | "21" | "21" | "star-square" | "121" | "121" | "star-square-open" | "221" | "221" |
    "star-square-dot" | "321" | "321" | "star-square-open-dot" | "22" | "22" | "star-diamond" | "122" | "122" |
    "star-diamond-open" | "222" | "222" | "star-diamond-dot" | "322" | "322" | "star-diamond-open-dot" | "23" | "23" |
    "diamond-tall" | "123" | "123" | "diamond-tall-open" | "223" | "223" | "diamond-tall-dot" | "323" | "323" |
    "diamond-tall-open-dot" | "24" | "24" | "diamond-wide" | "124" | "124" | "diamond-wide-open" | "224" | "224" |
    "diamond-wide-dot" | "324" | "324" | "diamond-wide-open-dot" | "25" | "25" | "hourglass" | "125" | "125" |
    "hourglass-open" | "26" | "26" | "bowtie" | "126" | "126" | "bowtie-open" | "27" | "27" | "circle-cross" | "127" |
    "127" | "circle-cross-open" | "28" | "28" | "circle-x" | "128" | "128" | "circle-x-open" | "29" | "29" |
    "square-cross" | "129" | "129" | "square-cross-open" | "30" | "30" | "square-x" | "130" | "130" | "square-x-open" |
    "31" | "31" | "diamond-cross" | "131" | "131" | "diamond-cross-open" | "32" | "32" | "diamond-x" | "132" | "132" |
    "diamond-x-open" | "33" | "33" | "cross-thin" | "133" | "133" | "cross-thin-open" | "34" | "34" | "x-thin" |
    "134" | "134" | "x-thin-open" | "35" | "35" | "asterisk" | "135" | "135" | "asterisk-open" | "36" | "36" | "hash" |
    "136" | "136" | "hash-open" | "236" | "236" | "hash-dot" | "336" | "336" | "hash-open-dot" | "37" | "37" |
    "y-up" | "137" | "137" | "y-up-open" | "38" | "38" | "y-down" | "138" | "138" | "y-down-open" | "39" | "39" |
    "y-left" | "139" | "139" | "y-left-open" | "40" | "40" | "y-right" | "140" | "140" | "y-right-open" | "41" | "41" |
    "line-ew" | "141" | "141" | "line-ew-open" | "42" | "42" | "line-ns" | "142" | "142" | "line-ns-open" | "43" |
    "43" | "line-ne" | "143" | "143" | "line-ne-open" | "44" | "44" | "line-nw" | "144" | "144" | "line-nw-open" |
    "45" | "45" | "arrow-up" | "145" | "145" | "arrow-up-open" | "46" | "46" | "arrow-down" | "146" | "146" |
    "arrow-down-open" | "47" | "47" | "arrow-left" | "147" | "147" | "arrow-left-open" | "48" | "48" | "arrow-right" |
    "148" | "148" | "arrow-right-open" | "49" | "49" | "arrow-bar-up" | "149" | "149" | "arrow-bar-up-open" | "50" |
    "50" | "arrow-bar-down" | "150" | "150" | "arrow-bar-down-open" | "51" | "51" | "arrow-bar-left" | "151" | "151" |
    "arrow-bar-left-open" | "52" | "52" | "arrow-bar-right" | "152" | "152" | "arrow-bar-right-open" | "53" | "53" |
    "arrow" | "153" | "153" | "arrow-open" | "54" | "54" | "arrow-wide" | "154" | "154" | "arrow-wide-open"

val symbol: Symbol = "diamond-wide"

println(symbol)

See also in scastie.

Output

Exception while compiling /tmp/scastie348582599853645637/src/main/scala/main.scala

  An unhandled exception was thrown in the compiler.
  Please file a crash report here:
  https://github.com/lampepfl/dotty/issues/new/choose

     while compiling: <no file>
        during phase: parser
                mode: Mode()
     library version: version 2.13.12
    compiler version: version 3.3.3
            settings: -bootclasspath /home/sbtRunnerContainer/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-library_3/3.3.3/scala3-library_3-3.3.3.jar:/home/sbtRunnerContainer/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.12/scala-library-2.13.12.jar -classpath /tmp/scastie348582599853645637/target/scala-3.3.3/classes:/home/sbtRunnerContainer/.ivy2/local/org.scastie/runtime-scala_3/1.0.0-SNAPSHOT/jars/runtime-scala_3.jar:/home/sbtRunnerContainer/.ivy2/local/org.scastie/api_3/0.30.0-SNAPSHOT/jars/api_3.jar:/home/sbtRunnerContainer/.cache/coursier/v1/https/repo1.maven.org/maven2/com/typesafe/play/play-json_3/2.10.0-RC5/play-json_3-2.10.0-RC5.jar:/home/sbtRunnerContainer/.cache/coursier/v1/https/repo1.maven.org/maven2/com/typesafe/play/play-functional_3/2.10.0-RC5/play-functional_3-2.10.0-RC5.jar:/home/sbtRunnerContainer/.cache/coursier/v1/https/repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-core/2.11.4/jackson-core-2.11.4.jar:/home/sbtRunnerContainer/.cache/coursier/v1/https/repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.11.4/jackson-annotations-2.11.4.jar:/home/sbtRunnerContainer/.cache/coursier/v1/https/repo1.maven.org/maven2/com/fasterxml/jackson/datatype/jackson-datatype-jdk8/2.11.4/jackson-datatype-jdk8-2.11.4.jar:/home/sbtRunnerContainer/.cache/coursier/v1/https/repo1.maven.org/maven2/com/fasterxml/jackson/datatype/jackson-datatype-jsr310/2.11.4/jackson-datatype-jsr310-2.11.4.jar:/home/sbtRunnerContainer/.cache/coursier/v1/https/repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.11.4/jackson-databind-2.11.4.jar -d /tmp/scastie348582599853645637/target/scala-3.3.3/classes -deprecation true -feature true

Stacktrace (of comparable compilation):

[error] java.lang.StackOverflowError
[error] dotty.tools.dotc.core.Types$Type.findMember(Types.scala:953)
[error] dotty.tools.dotc.core.ContextOps$.denotNamed(ContextOps.scala:38)
[error] dotty.tools.dotc.typer.Typer.loop$1(Typer.scala:458)
[error] dotty.tools.dotc.typer.Typer.findRefRecur$1(Typer.scala:521)
[error] dotty.tools.dotc.typer.Typer.findRef(Typer.scala:524)
[error] dotty.tools.dotc.typer.Typer.typedIdent(Typer.scala:585)
[error] dotty.tools.dotc.typer.Typer.typedNamed$1(Typer.scala:3087)
[error] dotty.tools.dotc.typer.Typer.typedUnadapted(Typer.scala:3196)
[error] dotty.tools.dotc.typer.Typer.typed(Typer.scala:3274)
[error] dotty.tools.dotc.typer.Typer.typed(Typer.scala:3278)
[error] dotty.tools.dotc.typer.Typer.typedAppliedTypeTree(Typer.scala:2227)
[error] dotty.tools.dotc.typer.Typer.typedInfixOp(Typer.scala:2999)
:
:

and from here on it comes into a loop

Remark

This error popped up in a real life automated code translation. An upper limit on the number of items in a type definition would be the best solution here i guess. There is always some limit.

[He-Pin]

Yes,but this should not be a stackoverflow, which may be rewritten in hand rolling stack

[SethTisue]

Try giving the compiler more stack (with -Xss)? The default JVM stack size is quite small; it's considered normal that unusual code may require additional stack at compile time.

[devlaam]

Well, this is a lot easier said then done. I use SBT for compilation. Following the manual and adding "-J-Xss16m" results in an sbt startup error (as does the original example in the manual). Leaving out the "-J" as is suggested here, does not make a difference. Any environment variable i define, like described here is stiffly ignored, as is information in the file ".sbtopts". Silent Build Tool ...

Some C₂H₅OH is needed for new idea's.

[OndrejSpanel]

I have used environment variable SBT_OPTS=-Xss500k to set compiler stack size successfully in sirthias/borer#608

See https://github.com/OndrejSpanel/borerScala3Error/blob/master/.github/workflows/scala.yml

[SethTisue]

a sampling of repos on my hard drive:

% grep Xss */.jvmopts
doobie/.jvmopts:-Xss8m
dotty.old/.jvmopts:-Xss1m
macrolizer/.jvmopts:-Xss2M
mdoc/.jvmopts:-Xss4m
metals/.jvmopts:-Xss4m
munit/.jvmopts:-Xss2m
sbt-scalafix/.jvmopts:-Xss4m
scala.212/.jvmopts:-Xss1M
scala.213/.jvmopts:-Xss1M
scala.dotty/.jvmopts:-Xss1M
scala3/.jvmopts:-Xss1m
scalafix/.jvmopts:-Xss4m
scalafx/.jvmopts:-Xss2M
scalameta/.jvmopts:-Xss4m
scalatest/.jvmopts:-Xss10M
scapegoat/.jvmopts:-Xss2M
shapeless/.jvmopts:-Xss4m
slick/.jvmopts:-Xss2M
zinc/.jvmopts:-Xss1M

% grep Xss */.sbtopts
diffx/.sbtopts:-J-Xss2M
interop-cats/.sbtopts:-J-Xss5M
playframework/.sbtopts:-J-Xss2M
scala-native/.sbtopts:#   -Xss is equivalent to -XXThreadStackSize
scala-native/.sbtopts:-J-Xss2M
squants/.sbtopts:-J-Xss2m
sttp/.sbtopts:-J-Xss2M
tapir/.sbtopts:-J-Xss2M

[SethTisue]

Following the manual and adding "-J-Xss16m" results in an sbt startup error (as does the original example in the manual)

What error is that? Can't help you with it without any details.

sbt -J-Xss16m works fine for me in my own repos.

[devlaam]

Although this thread is not meant for help on SBT I do appreciate the efforts.

ruud@raaf tyrosine $ sbt -J-Xss16m
[info] welcome to sbt 1.9.4 (AdoptOpenJDK Java 11.0.10)
[info] loading settings for project global-plugins from plugins.sbt ...
[info] loading global plugins from /.../.sbt/1.0/plugins
[info] loading settings for project tyrosine-build-build-build from metals.sbt ...
[info] loading project definition from /.../tyrosine/project/project/project
[info] loading settings for project tyrosine-build-build from metals.sbt ...
[info] loading project definition from /.../tyrosine/project/project
[success] Generated .bloop/tyrosine-build-build.json
[success] Total time: 1 s, completed 11 Apr 2024, 13:23:08
[info] loading settings for project tyrosine-build from metals.sbt,plugins.sbt ...
[info] loading project definition from /.../tyrosine/project
[success] Generated .bloop/tyrosine-build.json
[success] Total time: 0 s, completed 11 Apr 2024, 13:23:09
[info] loading settings for project root from build.sbt ...
[info] set current project to tyrosine (in build file:/.../tyrosine/)
[error] Expected symbol
[error] Not a valid command: -
[error] Expected end of input.
[error] Expected '--'
[error] Expected 'debug'
[error] Expected 'info'
[error] Expected 'warn'
[error] Expected 'error'
[error] Expected 'addPluginSbtFile'
[error] -J-Xss16m
[error]  ^

Btw, it gives the hint: "Expected '--'" but that gives almost the same result:

ruud@raaf tyrosine $ sbt --J-Xss16m
[info] welcome to sbt 1.9.4 (AdoptOpenJDK Java 11.0.10)
[info] loading settings for project global-plugins from plugins.sbt ...
[info] loading global plugins from /.../.sbt/1.0/plugins
[info] loading settings for project tyrosine-build-build-build from metals.sbt ...
[info] loading project definition from /...tyrosine/project/project/project
[info] loading settings for project tyrosine-build-build from metals.sbt ...
[info] loading project definition from /.../tyrosine/project/project
[success] Generated .bloop/tyrosine-build-build.json
[success] Total time: 1 s, completed 11 Apr 2024, 13:26:46
[info] loading settings for project tyrosine-build from metals.sbt,plugins.sbt ...
[info] loading project definition from /.../tyrosine/project
[success] Generated .bloop/tyrosine-build.json
[success] Total time: 0 s, completed 11 Apr 2024, 13:26:48
[info] loading settings for project root from build.sbt ...
[info] set current project to tyrosine (in build file:/.../tyrosine/)
[error] Expected symbol
[error] Not a valid command: --
[error] Expected 'debug'
[error] Expected 'info'
[error] Expected 'warn'
[error] Expected 'error'
[error] Expected 'addPluginSbtFile'
[error] --J-Xss16m
[error]   ^

and without the -J I get the same as the first error.

[OndrejSpanel]

Do not enter it as a command line option, it does not work that way. It needs to be written in the .sbtopts or .jvmopts (each of them uses a different syntax) or given in the environment variable SBT_OPTS.

[SethTisue]

@devlaam i wonder if your sbt script is the standard one

[devlaam]

Do not enter it as a command line option, it does not work that way

Well, @OndrejSpanel, it should according to the manual and the experiences of @SethTisue.

It needs to be written in the .sbtopts or .jvmopts (each of them uses a different syntax) or given in the environment variable SBT_OPTS.

I tried export SBT_OPTS="-Xss16m" (with this procedure, and checked with env to see if the variable was set) and by defining a .sbtopts file (and checked with cat .sbtopts). As far as i can tell, those settings are ignored by sbt, and the crash still happens.

i wonder if your sbt script is the standard one

Yeah, that's a good question. The only thing I know is that I did not modify it.

However, I now also tried export JAVA_OPTS="-Xss16m", which seems to do the trick. With this value the original file compiles.

This leaves me with the question:

• Is this still a bug? (The crash report explicitly requested me to file a bug report)
• Should we recommend users to use different default settings (I also run out of metaspace sometimes)?
• Should the crash report text be changed to something like: "Increase your stack size first, before filing a bug report."?
• Should we introduce a warning for types that are too long for default settings?