Transform/explicit outer #171

odersky · 2014-09-06T02:51:18Z

Explicit outer is now enabled. Some fairly extensive changes were needed to make it happen. In particular, we had to bring back the inSuperCall mode bit. Review by @DarkDimius

These were not handled before.

Added decorators for symbols that can query specific annotations and annotation arguments (for now, -deprecated and -migration are added)

Added method to traverse all parts of a type.

Two improvements to TreeTransform: 1) Added transformOther functionality which handles trees not handled by other parts 2) Passes down Mode.Pattern in the context when in a pattern. TreeTransform no longer normalizes unknown trees but passes them to transformOther. The former Companions phase has been renamed to FirstTransform. It now performs the following optimizations: - adds companion objects (this was done before) - other normalizations that were formally done in TreeTransform, - rewrite native methods to stubs (this was formally done in RefChecks)

Cycles are now detected early, when an info is first completed. Legal, f-bounded cycles are broken by a LazyRef, which will construct its type lazily. This makes checkBounds validation of AppliedTypeTrees work (in FirstTransform). Formerly, this stackoverflowed despite the laziness precautions in findMember. Todo: Do the same for class files coming from Java and Scala 2.x.

Insert LazyRefs to break cycles for F-bounded types that are unpickled or read from Java signatures.

Now that F-bunded types are treated more robustly, we can check bounds for non-emptyness during Typer. This unvealed one wrong test (wonder how that passed scalac?), which got moved to neg.

Statements are now transformed with the transform returned by prepareStats, analogoys to the other prepare methods.

This is still disabled, because the prepare machinery in transform does not work yet. Concretely, prepare ops need to return a new TreeTransform but that tree transform has an undefined phaase id. We need some architectural changes to disentangle transforms from phases.

... so that it can be combined with TreeTransform in a trait composition in a future dientanglement of TreeTransforms and Phases.

Would be flagged as unimplemented members in refChecks otherwise

TreeTransforms are no longer phases. This allows to generate new transforms in prepare... methods without running into the problem that thee new transforms are undefined as phases. It also makes for a cleaner separation of concerns.

Should have lazy flag set, otherwise forward reference checking would fail for modules. Note: LazyVals needed to be disabled because it also should transform module vals, but didn't do this so far because it only tested the Lazy flag. It turned out the module val transformation exposed some bugs in lazy vals in that LazyVals creates symbols as a side effect and enters them into scopes. Such mutations are allowed onyl in very specific cases (essentially only for local throw-away scopes).

Used to show "mutable var x: T".

Scopes are also used in overriding pairs, and there multiple types with the same name can be entered in a scope. So the assert to the contrary should be limited to typechecking only.

Eta-lifting picked some arbitrary base type. It turned out that i94-nada failed once we add a product trait to case classes (in the next commit) because Eta-Kifting picked Product as the base type, even though the target type was bounded by Monad. We now change the scheme so that the target type is included in the lifting, in order to avoid that we lift to useless types.

Case classes with arity <= 1 now also get a ProductN parent trait. This is necessary because we inherit productArity and Element methods in case classes from the ProdictN trait. Needed to add Product0 for this, which is not defined in Scala2.x.

After erasure, former Any members become Object members. Also, fixed some typos and added some TODOs on addBridges.

Was missing before.

asInstance/isInstance/ensureConforms/and/or. They replace some former "mk..." methods.

Also renamed Boolean_and/or to _&&/||, to make it conform to naming convention for other Definition operators.

RefChecks needs both methods.

New phase for Synthetic Method generation. Scala 2.x did it in Typer, but it's cleaner to do it in a separate phase.

If a TypeRef with an expanded name is an alias type, print the alias instead.

Now that caes classes always inherit from ProductX, we can avoid the special case. (We need to define _1 anyway to implement Product1).

(1) set position of companion object def (2) companions of case classes taking multiple parameter lists do not inherit from a function type (reason: we can't straightforwardly converyt a curried method with multiple parameter lists to a function value).

For overriding checks we need a concept where a val can match a def. Normal matches does not provide this.

1) Constructors of inner classes get outer parameters 2) Outer arguments are passed as needed.

Also regorganized flags a bit to better use available slots.

A this reference hide in a self ident, and be subsequently missed when deciding on whether outer accessors are needed and computing outer paths. We do this normalization directly after Typer, because during typer the ident should rest available for hyperlinking.

Makes the method pioneered in elidablePrefix more general. Also applies it to accessWithin.

Outer paths, parameters and arguments are inserted in erasure, using methods provided by OuterAccessors. Also fixed a stupid bug in Erasure#constant

Needed to keep a record of definitions in supercall arguments. These may not see the enclosing class.

If there's a crash, we always want to see where the retyper was, not just for tree checking.

Syntactically enclosing class is not accessible from such definitions, so should be skipped.

Partial revert of 08c6eac "this type is a term ref to the source module". The problem with doing this is that it introduces spurious outer references. An inner module that contains self referenves always needs the directly enclosing class. The revert avoids this dependency by making ThisTypes always point to TypeRefs. Several other changes were necessary to make the builds pass: TypeRefs had to get prefixes after erasure so that they can be reloaded. Symbols of such typerefs had to be retrieved without forcing a denotation. One test (blockescapes.scala) fails and is moved to pending, awaiting further resolution. Also two other new tests in pending which currently fail (and have failed before).

Module roots were mis-characterized, which meant that module symbols were loaded twice.

Have a configurable printer to which cyclic error messages are sent.

Catch exceptions and embed into string instead of passing exception on. Reason: i"" strings are for diagnostic output but may cause exceptions such as CyclicReferences, stale symbols and so on. We never want to crash the program with such an exception.

... by reverting a premature optimization in Erasure.

Better to keep the old name for easy cross-referencing with Scala 2.

Now also testing that after erasure no outer this exists. Tests suit now includes calls to local classes and methods which need an outer pointer, as well as passing an outer pointer along a secondary constructor.

DarkDimius · 2014-09-06T07:58:52Z

Lets concentrate on this and get it merged(patmat depends on explicitOuter as it needs outer references). @odersky tell me if you're done, I'll hijack this branch, add todo comments from discussion and merge it.

Have a general way how a phase can establish a postcondition which will be checked each time a later phase is tree-checked. Moves erasure constraints from TreeChecker to Erasure's post condition.

foreachSubTree and existsSubTree are now infix methods. Streamlines their use somewhat.

But allow pattern matching to provide outer accessors when needed using with ensureOuterAccessors.

odersky · 2014-09-06T16:23:53Z

I am done now. Regarding interaction with pattern matcher, see last commit.
The idea is that the pattern matcher should tell explicit outer of any
outer links that need to be maintained because of pattern matching.

But explicit outer depends on erasure, so probably we have to get the whole
long commit queue in. Which would be a good thing IMO.

On Sat, Sep 6, 2014 at 9:58 AM, Dmitry Petrashko notifications@github.com
wrote:

Lets concentrate on this and get it merged(patmat depends on explicitOuter
as it needs outer references). @odersky https://github.com/odersky tell
me if you're done, I'll hijack this branch.

—
Reply to this email directly or view it on GitHub
#171 (comment).

Martin Odersky
EPFL

DarkDimius · 2014-09-06T21:53:05Z

Regarding interaction with pattern matcher, see last commit. The idea is that the pattern matcher should tell explicit outer of any outer links that need to be maintained because of pattern matching.

I believe that pattern matcher should be after explicit outer for two reasons:

it already needs to reffer to $outer symbol in order to make isInstanceOf checks on it. (in scalac ability to use this is implemented through accessing non-existent <outer>(with brackets) field)
it cannot ever say that an outer link can be removed, as, not having closed world assumption, any non-private class can be used in pattern matching outside current compilation unit

DarkDimius · 2014-09-06T22:09:04Z

@odersky If you're reading by mail please see my updated comment.

I believe that pattern matcher should be after explicit outer for two reasons:

odersky · 2014-09-07T02:15:47Z

But how does explicit outer know what gets referred to in the patten matcher?

I see two possibilities:

We add an explicit outer field even for private classes that need none (i.e. LocalAccessFlags is just Module not Module | Private.
We let the pattern matcher run before ExplicitOuter and have it create outer fields with ensureOuterAccessors.

Either way works for me. 1) is a little bit more efficient. 2) is a little bit simpler.

DarkDimius · 2014-09-07T04:21:20Z

@odersky, ok, I see your point now. Let's go with 2.

Backport "Do not return java outline dummy constructor in `primaryConstructor`" to 3.3 LTS

odersky added 30 commits August 3, 2014 20:43

Code to handle overloaded unapply/unapplySeq methods

2020938

These were not handled before.

Added version settings -migration, -source

168e4f1

Annotation decorators for symbols

e25232d

Added decorators for symbols that can query specific annotations and annotation arguments (for now, -deprecated and -migration are added)

Type#foreachPart

85044e4

Added method to traverse all parts of a type.

LazyRefs break cycles for unpickled types

058729c

Insert LazyRefs to break cycles for F-bounded types that are unpickled or read from Java signatures.

Re-enabled checkbounds tests

19b6a04

Now that F-bunded types are treated more robustly, we can check bounds for non-emptyness during Typer. This unvealed one wrong test (wonder how that passed scalac?), which got moved to neg.

prepareStats should span all statement transforms

9ec3a4f

Statements are now transformed with the transform returned by prepareStats, analogoys to the other prepare methods.

Made Phase a trait

80ee8b1

... so that it can be combined with TreeTransform in a trait composition in a future dientanglement of TreeTransforms and Phases.

Fixed fully qualified name of migration annotation

bb1972b

ParamAccessors are assumed to have definition

9024f25

Would be flagged as unimplemented members in refChecks otherwise

Disentangle phases from treetransforms

57c6c85

TreeTransforms are no longer phases. This allows to generate new transforms in prepare... methods without running into the problem that thee new transforms are undefined as phases. It also makes for a cleaner separation of concerns.

In refined printer, show only source modifier flags.

80b866f

Used to show "mutable var x: T".

Narrow assertion for Scope#enter

bbf777a

Scopes are also used in overriding pairs, and there multiple types with the same name can be entered in a scope. So the assert to the contrary should be limited to typechecking only.

Fix Object's scope after ersure

d388e94

After erasure, former Any members become Object members. Also, fixed some typos and added some TODOs on addBridges.

Add CaseAccessor flag for case accessors

b89c4af

Was missing before.

Added some more methods as infix tree operations:

c7f817f

asInstance/isInstance/ensureConforms/and/or. They replace some former "mk..." methods.

New methods in Definitions

98d2583

Also renamed Boolean_and/or to _&&/||, to make it conform to naming convention for other Definition operators.

Sperate matchingDecl and mathingMember.

ab63413

RefChecks needs both methods.

Add SyntheticMethods miniphase

ce53589

New phase for Synthetic Method generation. Scala 2.x did it in Typer, but it's cleaner to do it in a separate phase.

Avoid printing expanded names

974f706

If a TypeRef with an expanded name is an alias type, print the alias instead.

Use _1 intstead of get for accessing unary case class parameters

a3cc938

Now that caes classes always inherit from ProductX, we can avoid the special case. (We need to define _1 anyway to implement Product1).

Fix a problem due to different type inference

81745e2

Two fixes in desugar

843f976

(1) set position of companion object def (2) companions of case classes taking multiple parameter lists do not inherit from a function type (reason: we can't straightforwardly converyt a curried method with multiple parameter lists to a function value).

Add matchesLoosely

472d711

For overriding checks we need a concept where a val can match a def. Normal matches does not provide this.

odersky added 15 commits August 31, 2014 19:34

Add outer parameters in constructor calls.

ec1d274

1) Constructors of inner classes get outer parameters 2) Outer arguments are passed as needed.

Added OuterAccessor flag.

d85f2dd

Also regorganized flags a bit to better use available slots.

Allow more leniency in time travel.

2788547

Makes the method pioneered in elidablePrefix more general. Also applies it to accessWithin.

Completed outer paths.

2558c49

Outer paths, parameters and arguments are inserted in erasure, using methods provided by OuterAccessors. Also fixed a stupid bug in Erasure#constant

Add inSuperCall mode and flag.

2251026

Needed to keep a record of definitions in supercall arguments. These may not see the enclosing class.

Move crash diagnostics from TreeChecker to ReTyper.

431288d

If there's a crash, we always want to see where the retyper was, not just for tree checking.

Enclosing class skips classes from inSuperCall definitions

652a7e5

Syntactically enclosing class is not accessible from such definitions, so should be skipped.

Fix a bug in unpickler

b33f604

Module roots were mis-characterized, which meant that module symbols were loaded twice.

CyclicError messages now go to printer

795796d

Have a configurable printer to which cyclic error messages are sent.

Bring back blockescapes test...

a661bed

... by reverting a premature optimization in Erasure.

Rename OuterAccessors to ExplicitOuter

44bdec1

Better to keep the old name for easy cross-referencing with Scala 2.

Better tests and more fixes for ExplicitOuter

e104093

Now also testing that after erasure no outer this exists. Tests suit now includes calls to local classes and methods which need an outer pointer, as well as passing an outer pointer along a secondary constructor.

odersky added 3 commits September 6, 2014 11:36

Generalize phase postcondition checking.

b58b906

Have a general way how a phase can establish a postcondition which will be checked each time a later phase is tree-checked. Moves erasure constraints from TreeChecker to Erasure's post condition.

Make foreach/exsists into tree infix methods

c3078b1

foreachSubTree and existsSubTree are now infix methods. Streamlines their use somewhat.

Move explicit outer after pattern matching

b129332

But allow pattern matching to provide outer accessors when needed using with ensureOuterAccessors.

DarkDimius mentioned this pull request Sep 9, 2014

Patmat #172

Closed

DarkDimius added a commit to dotty-staging/dotty that referenced this pull request Sep 9, 2014

Changes from discussion in scala#171

3f7945b

DarkDimius added a commit to dotty-staging/dotty that referenced this pull request Sep 17, 2014

Changes from discussion in scala#171

f0dfea4

odersky mentioned this pull request Sep 17, 2014

Pattern matcher #174

Merged

DarkDimius merged commit b129332 into scala:master Oct 11, 2014

WojciechMazur pushed a commit to WojciechMazur/dotty that referenced this pull request Mar 19, 2025

Merge pull request scala#171 from scala/backport-lts-3.3-22104

79699cf

Backport "Do not return java outline dummy constructor in `primaryConstructor`" to 3.3 LTS

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Transform/explicit outer #171

Transform/explicit outer #171

Uh oh!

odersky commented Sep 6, 2014

Uh oh!

DarkDimius commented Sep 6, 2014

Uh oh!

odersky commented Sep 6, 2014

Uh oh!

DarkDimius commented Sep 6, 2014

Uh oh!

DarkDimius commented Sep 6, 2014

Uh oh!

odersky commented Sep 7, 2014

Uh oh!

DarkDimius commented Sep 7, 2014

Uh oh!

Uh oh!

Transform/explicit outer #171

Transform/explicit outer #171

Uh oh!

Conversation

odersky commented Sep 6, 2014

Uh oh!

DarkDimius commented Sep 6, 2014

Uh oh!

odersky commented Sep 6, 2014

Uh oh!

DarkDimius commented Sep 6, 2014

Uh oh!

DarkDimius commented Sep 6, 2014

Uh oh!

odersky commented Sep 7, 2014

Uh oh!

DarkDimius commented Sep 7, 2014

Uh oh!

Uh oh!