private val referred to by a WeakReference should still create a field

[Arthurm1]

In Scala 2, private variables in a class would be defined in the class. In Scala 3 they seem to be defined in the constructor.

I ran into this because I had weak references to the variables and then the variables got garbage collected - which was not what I wanted.

Compiler version

3.1.0-RC3

Minimized code

class Foo(i: Int) {
  private val bar: Int = 0
}

seems to give the equivalent in Java of...

public class Foo
{
    Foo(int i) {
        int bar = 0;
    }
}

when in Scala 2 it gave...

public class Foo
{
    private int bar = 0;

    Foo(int i) {
    }
}

I think Scala 2 also defined private[this] val in the class, not the constructor.

[s5bug]

Copying from the Scala Discord, adding a scope to private:

class Foo(i: Int) {
  private[Foo] val bar: Int = 0
}

gives

public class Foo {
    private final int bar;

    public Foo(int i) { this.bar = 0; }

    public int bar() { return this.bar; }
}

in Scala 3. (This is not an answer, just additional info)

[smarter]

This is intentional, we don't create class fields for unused private val/var, you can either make them package-private or have a (possibly private) def refer to them.

[Arthurm1]

So having a private def refer to bar is the only way to force it to be a class field without Scala automatically creating a public getter? There's no special keyword to force it to be a class field, e.g. private[class] val bar:int = 0?

My issue is that I have something like this...

class Foo(i: Int) {
  private val privateBar: Int = 0
  val publicRef = new java.lang.ref.WeakReference(privateBar)
}

which was fine in Scala 2 but now publicRef.get returns null after a GC.

[smarter]

Ah right, WeakReference is magic, so maybe it should be special-cased by the compiler.

[smarter]

For anyone wanting to work on this, the logic which decides which private val should not be moved to the constructor is here: https://github.com/lampepfl/dotty/blob/7cb30029a5eadea5d4848519f3ae96aca319d914/compiler/src/dotty/tools/dotc/transform/Constructors.scala#L49-L80

[sir-wabbit]

I can imagine there are situations where you don't want a private val to become an actual field and you still have a public WeakReference to it. Say something like

class BarAllocator(barManager: BarManager) {
  private val privateBar = new Bar
  val publicRef = new java.lang.ref.WeakReference(privateBar)
  barManager.add(privateBar) // may or may not delete it later
}

It seems to me that adding special cases for special cases is a bad idea and a consistent and simple set of rules that doesn't randomly violate one's expectations is more preferable. Adding a private def seems like a reasonable workaround assuming that compiler behavior is documented somewhere and stays consistent so that the code doesn't suddenly break.

[Arthurm1]

Well this has changed from Scala 2 where the field would have been defined in the class, not the constructor.

But it would be nice if ambiguity could be removed, and it was possible to be explicit via a keyword rather than have to create private def for every field you want to ensure is local to the class.

e.g.

private[class] val foo
private[constructor] val bar

[ghostbuster91]

As the next issue spree is coming I would like to clarify the objectives in that task.

At the beginning I came to the conclusion that we should follow @smarter's suggestion and add a special way of handling values which are referenced by a member with WeakReference type. I came up with following motives to support that approach:

• code which was valid in scala2 and is still valid in scala3 behaves the same
• there should be a way to make this code behave correctly in scala3 without a need to use private[Foo] which comes with additional cost of unnecessary method declaration

However, the more I think about that the more I am in doubts whether this is possible at all.
Below are some potential obstacles which seem to be relevant:

• WeakReference is not a final class, which means that it can be extended by the end user. Even in the jdk alone there are types which extend it (ThreadLocal.ThreadLocalMap.Entry, WeakHashMap.Entry). That alone might not be an issue as we could check whether the WeakReference type is accessible from given class hierarchy. Still, it is something which we need to keep in mind.
• there is no option to truly knows if something was passed to a weakReference. Here are some problematic scenarios:

  class Foo(i: Int) {
   private val privateBar: Int = 0
   val publicRef = Foo.wrap(privateBar) // The return type of this method is a weakReference 
                                      // but we really don't know whether that parameter was indeed passed there.
  }
  
  object Foo {
   def wrap(i: Int) : java.lang.ref.WeakReference[Int] = new java.lang.ref.WeakReference(i)
  }

  class Foo(i: Int) {
   private val privateBar: Int = 0
   private val privateBar2: Int = 0
   private val privateBar3: Int = 0
   val publicRef = Foo.wrap(privateBar, privateBar2, privateBar3) // Here i2 is only used to determine 
                                                                   // which variable to hold in the weakReference
  }
  
  object Foo {
   def wrap(i: Int, i2: Int, i3: Int) : java.lang.ref.WeakReference[Int] = 
     if(i == 0) new java.lang.ref.WeakReference(i2) 
     else new java.lang.ref.WeakReference(i3)
  }

  class Foo(i: Int) {
    private val privateBar: Int = 0
    val bar = new Bar(privateBar)
  }
  
  class Bar[A](a: A){
    val weakRef = new java.lang.ref.WeakReference(a)
  }

Given that, I don't think that it is feasible to add any special handling for the WeakReference as it will be error-prone and confusing.

The remaining question is whether to add a way to allow developers mark some fields as to be retained, or whether we are fine with the workaround of using private[Foo] with the cost of additional method declaration.

[odersky]

You can force a field by writing

private[Foo] val privateBar: Int = 0

Is that not enough?