How to document the API of compiler intrinsic definitions

[bishabosha]

there are several standard library public definitions that do not appear in http://dotty.epfl.ch/api/index.html because they are generated on the fly by the compiler and so do not have a definition in tasty files

Magic in Scala 2/3

the following appear in https://www.scala-lang.org/api/2.13.4/index.html:

• scala.Nothing
• scala.AnyVal
• scala.Any
• scala.Singleton
• scala.Null

Magic in Scala 3

• scala.AnyKind
• scala.Matchable
• scala.&
• scala.|

Miscellaneous

• root package (Edit: yes I see now that the root is just the API page but just not labelled as root)

[abgruszecki]

root package is, I believe, already documented. As for the other definitions, we could either create special uncompilable source files and add a way to document source files as well (feasible, though it'd require diverging away from TastyInspector) or add a way to load docstrings for some symbols from text files and add a way of forcing a class to be documented even though it wasn't loaded from Tasty.

EDIT: the comparison of the two approaches is that in the first, we always have deftrees for every symbol we document, which is more in line with how the doctool thinks; the second approach might have applications beyond documenting stdlib symbols - for instance, one may load multiple libraries to document and add custom docstrings to symbols not originally documented. Now that I think about it, perhaps we could actually expose functionality for attaching docstrings to symbols on Reflect level? Could be useful, and is feasible given that docstrings are kept in a mutable hashmap.

[smarter]

or add a way to load docstrings for some symbols from text files

or add the docstrings to the symbols when we create them in Definitions.scala, that way :doc in the repl will work on them too, etc.

[BarkingBad]

The solution proposed by smarter seems to be the most reasonable.

[BarkingBad]

After a little research I see few options:

Reusing files from library-aux

We could try compiling these classes to get TASTY files and attach it to the doctool run. However, it is impossible right now; dotty prevents shadowing scala.Any and other 4 fq names. Also, we cannot have final trait Singleton in dotty, as these keywords are excluding each other.
Pros:

• We reuse code for Scala 2 and 3

Cons:

• We won't have these docstrings loaded for symbols in e. g. REPL
• We need to make compilation pass on these sources (get rid of final trait or find some workaround)

At last we could do some file processing (change package for sake of not conflicting with fq names, remove final keyword for trait) after we set up them in our sbt run, but this is very unsafe solution.

New docstrings

We can either:

• have them declared in separate text files with some kind of annotation to match them
• have them declared in pseudo scala files (like in Scala2) and try to compile them with dotty
• have them embedded into code, f. e. properties or map Symbol -> Comment

Pros:

• easy to implement
• the last solution will be available in REPL

Cons:

• the first two solutions will not be available in REPL
• documentation for Scala 2 and 3 sources can eventually diverge as someone will change them.

@smarter do you have any opinions what solution fits the best?

[smarter]

Embedding them in the code still looks like the best solution to me here.