Allow closures to mutate their upvars

[rntz]

Rust does not permit the following code:

pure fn generate_nats_from(x_: uint) -> pure fn@() -> uint {
  let mut x = x_;
  || { x += 1; x }
}

It is disallowed for two reasons:

1. You cannot implicitly capture a mutable variable, you need a capture clause.
2. You cannot assign to captured mutable variables from heap closures.

These restrictions appear to primarily have historical motivations: originally, local variables were all mutable and were captured by copying their values at the time the closure was created.

However, it is often convenient to create a "stateful" heap closure, which maintains some hidden local state that may change between calls to it. The most natural way to do this is to allow heap closures to capture mutable variables by reference.

If we allowed this, then generate_nats_from(0) would return a function f such that the first call to f returned 1, the second 2, the third 3, etc. This is very similar to a python-style generator, albeit written explicitly instead of with yield.

To achieve this currently in rust, we must explicitly allocate a mutable managed pointer, like so:

fn generate_nats_from(x_: uint) -> fn@() -> uint {
  let x = @mut x_;
  || { *x += 1; *x }
}

Notice that I had to drop the pure annotation on the returned function, because it mutates an @-pointer. However, the function is notionally pure: it mutates nothing visible to its caller, only its own internal state.

It also makes the code uglier, and in principle less efficient, to do it this way. I say "in principle less efficient" because the code given allocates a separate GC'd box for x, when it could instead become part of the environment of the closure we return. Actually implementing this optimization, however, might be difficult.

Mostly I want this feature because it makes writing stateful closures nicer, and it makes writing pure stateful closures possible.

For examples of code that could benefit from this, see https://github.com/rntz/rust-sandbox/blob/master/stream.rs , in particular https://github.com/rntz/rust-sandbox/blob/905de30e5a7af8422ea3604b90f6443745bf6233/stream.rs#L30 and https://github.com/rntz/rust-sandbox/blob/905de30e5a7af8422ea3604b90f6443745bf6233/stream.rs#L101 .

Note in particular that the only thing keeping most of the functions in that module, and the definition of the Stream type itself, from being pure is this issue. As far as I can tell, lazy streams of this sort are doomed to second-class citizenship in the Rust purity system as long as this is not permitted.

[brson]

What about using ~mut foo to hold that mutable state?

fn generate_nats_from(x_: uint) -> fn@() -> uint {
  let x = ~mut x_;
  || { *x += 1; *x }
}

With type-based move (on incoming) it shouldn't require a capture clause to move x into the closure.

If the allocation is a problem one could have a struct with a mutable field. Of course, mutable fields are probably going away, making that not work.

If moves based on type also applied to mutable types and you could mutate captures, then your original example could work as written.

[brson]

@pcwalton says mutable types also move by default, so seemingly the only missing piece is to allow mutable closure environments.

[nikomatsakis]

It turns out that we really want this as part of the plan to remove mut fields. After discussing with @pcwalton I think we formulated a plan as follows: you can capture a mut variable, but it moves the value in and also prevents it from being later reassigned. That means that once you capture the mut variable it will only be accessible (and mutable) from within the closure. This should be a small tweak to liveness.

[nikomatsakis]

I should add that this should be discussed in a group meeting, I think.

[catamorphism]

We never had consensus here, so I'm going to courageously close this. Of course, someone else could always courageously reopen it.

[nikomatsakis]

I don't know that we ever discussed this in detail? I think it's an important feature, though, particularly if we want to avoid the need for a type like Cell (which I do).

[catamorphism]

@nikomatsakis No, we didn't discuss it -- I reasoned that if there had been sufficient interest, the discussion would have happened by now :-) Perhaps we can put it on the agenda for the next meeting.

[nikomatsakis]

Updated title. I consider this a sub-piece of #2202

[metajack]

Nominated the meta bug.

[huonw]

Triage: don't really understand this. cc #9351 since it can be used as a work around for this issue.

[alexcrichton]

Closing, @fn() has been gone for quite a long time. I also think that this is not quite as relevant any more.