Experimental annotation behaviour redesign

[prolativ]

We need to rethink how @experimental should work to:

• fix the problems with bootstrapping the compiler (until then 3.0.0 has to be used as the reference compiler)
• eliminate strange behaviours and unexpected limitations (e.g. Experimental trait cannot be instantiated anonymously #13091)
• take into account the expectations of the community regarding this feature

Related discussion: https://contributors.scala-lang.org/t/experimental-annotation-behaviour-redesign/5187

[odersky]

I think @experimental should behave like erased. To refer to an identifier that is annotated @experimental you need to be either in a definition marked @experimental or you need to run a snapshot compiler. That's it. I don't see the need to check more than that.

That would solve the bootstrap problem since tests of experimental features can be marked experimental themselves.

[prolativ]

As mentioned here https://contributors.scala-lang.org/t/experimental-annotation-behaviour-redesign/5187/7?u=prolativ
it seems to me that we should distinguish here between experimental features of the compiler and experimental parts of APIs of libraries, as in the latter case the requirement of using a snapshot compiler doesn't seem to make sense. Unless @experimental was not designed with this use case in mind at all and it's always supposed to mean "directly or transitively using experimental compiler features".

[prolativ]

@odersky I believe the bootstrapping problem would be solved in this case only if we allowed compiling experimental definitions with a non-snapshot compiler but not using experimental features.

[smarter]

in the latter case the requirement of using a snapshot compiler doesn't seem to make sense

We require a snapshot compiler to use experimental APIs in the standard library because we want to be able to change or remove these APIs at anytime without breaking binary and tasty compatibility for non-experimental code. Forcing the use of a snapshot compiler for experimental APIs ensure that even if someone publishes a library that uses these APIs, a project that uses a release compiler cannot accidentally depend on them, because the tasty version will be different (we always have a non-zero ExperimentalVersion for nightly builds).

[prolativ]

What I meant is whether when someone designs the API of their library and they introduce e.g. some method whose signature is quite likely to change in the future, is it OK to annotate such method with @experimental even if it doesn't use any experimental features of the compiler? Then @experimental be something kind of dual to @deprecated.
Or is @experimental ONLY for using/declaring experimental features of the compiler?

[smarter]

So far we've only been considering @experimental for compiler usecases, it could also be useful for external libraries but that seemed like something that should be discussed with library authors and iterated on first before becoming official which is why using @experimental itself was made experimental.

[dwijnand]

Just realised: even if we did rework the checks so they only checked usages of experimental APIs, rather than the definition of experimental APIs, the fact that @experimental is experimental means those definitions (e.g. in scala3-library) need a snapshot compiler because they're using @experimental.

Perhaps we really should do like Rust and give the compiler a backdoor so a non-snapshot compiler can bootstrap...

[prolativ]

So maybe we could allow using experimental features inside definitions which are experimental themselves no matter what compiler we use but usage of experimental features outside of experimental context would require using a snapshot compiler?

[odersky]

@dwijnand Maybe @experimental should not be experimental?

@prolativ

So maybe we could allow using experimental features inside definitions which are experimental themselves no matter what compiler we use but usage of experimental features outside of experimental context would require using a snapshot compiler?

That's exactly what I proposed.

[odersky]

For language imports: You can treat language.experimental as @experimental. That is, you can import experimental features with a snapshot compiler or inside an @experimental definition. Maybe you want to customize the error message if the condition is not satisfied.