Do we need Send bounds to stabilize async_fn_in_trait?

[tmandry]

Problem: Spawning from generics

Given an ordinary trait with async fn:

#![feature(async_fn_in_trait)]

trait AsyncIterator {
    type Item;
    async fn next(&mut self) -> Option<Self::Item>;
}

It is not currently possible to write a function like this:

fn spawn_print_all<I: AsyncIterator + Send + 'static>(mut count: I)
where
    I::Item: Display,
{
    tokio::spawn(async move {
        //       ^^^^^^^^^^^^
        // ERROR: future cannot be sent between threads safely
        while let Some(x) = count.next().await {
            //              ^^^^^^^^^^^^
            // note: future is not `Send` as it awaits another future which is not `Send`
            println!("{x}");
        }
    });
}

playground

Speaking more generally, it is impossible to write a function that

• Is generic over this trait
• Spawns a task on a work-stealing executor
• Calls an async fn of the trait from the spawned task

The problem is that the compiler does not know the concrete type of the future returned by next, or whether that future is Send.

Near-term mitigations

Spawning from concrete contexts

However, it is perfectly fine to spawn in a non-generic function that calls our generic function, e.g.

async fn print_all<I: AsyncIterator>(mut count: I)
where
    I::Item: Display,
{
    while let Some(x) = count.next().await {
        println!("{x}");
    }
}

async fn do_something() {
    let iter = Countdown::new(10);
    executor::spawn(print_all(iter)); // <-- This works!
}

playground

This works because spawn occurs in a context where

• We know the concrete type of our iterator, Countdown
• We know the future returned by Countdown::next is Send
• We therefore know that the future returned by our call to print_all::<Countdown> and passed to spawn is Send

Making this work smoothly depends on auto trait leakage.

Adding bounds in the trait

Another workaround is to write a special version of our trait that is designed to be used in generic contexts:

#![feature(return_position_impl_trait_in_trait)]

trait SpawnAsyncIterator: Send + 'static {
    type Item;
    fn next(&mut self) -> impl Future<Output = Option<Self::Item>> + Send + '_;
}

playground

Here we've added the Send bound by using return_position_impl_trait_in_trait syntax. We've also added Self: Send + 'static for convenience.

For a trait only used in a specific application, you could add these bounds directly to that trait instead of creating two versions of the same trait.

For cases where you do need two versions of the trait, your options are

• If you control both versions of the trait, write a blanket impl that forwards from SpawnAsyncIterator to AsyncIterator (playground)
• Write a macro that expands to a delegating impl for a given type
• Write impls by hand for each type, depending on what you need

Only work-stealing executors

Even though work-stealing executors are the most commonly used in Rust, there are a sizable number of users that use single-threaded or thread-per-core executors. They won't run into this problem, at least with Send.

Aside: Possible solutions

Solutions are outside the scope of this issue, but they would probably involve the ability to write something like

fn spawn_print_all<I: AsyncIterator<next(): Send> + Send + 'static>(mut count: I)
//                                 ^^^^^^^^^^^^^^ new (syntax TBD)
where
    I::Item: Display,
{
    ...
}

Further discussion about the syntax or shape of this solution should happen on the async-fundamentals-initiative repo, or a future RFC.

Questions

• How often do people see this problem in practice?
• Is this problem, despite the mitigations, bad enough that we should hold back stabilization of async_fn_in_trait until we have a solution?

If you've had a chance to give async_fn_in_trait a spin, or can relay other relevant first-hand knowledge, please comment below with your experience.

[withoutboats]

(NOT A CONTRIBUTION)

I just read the blog post and I thought it would be a useful datapoint that I literally wrote code (using async-trait) that would have run into this issue earlier today. I spawn in generic code all the time; a common case is that there is some kind of dependency injection in order to mock code that would use the network for testing purposes; such code is an obvious use case for an async method (EDIT: and there are many other reasons to use generics for dependency injection, eg supporting multiple different transport protocols)

[Ralith]

The async code I write is rarely generic, rarely spawns, and the combination is hence rare indeed. Although there are undoubtedly cases where this capability is a blocker, I think async traits are a large step forwards in usefulness without them, and it doesn't seem like the straightforward syntax proposed here will need breaking revision in the future.

[Sherlock-Holo]

the unsend future return by async trait is unusually, in the past people use the async trait crate and they are familiar that the future is send, if not, will mark at the trait as an attribute, and people use tokio or async std runtime in most cases, both of them are work stealing runtime, which require the future is send

[jplatte]

I tried using this in axum a week or two ago. Because it does type erasure, it can only support either Send futures or non-Send futures and obviously the former is what most people want. This feature doesn't allow that restriction though and so I couldn't use it.

Edit: just saw the RPITIT mention in the blog post and I think that didn't work when I tried it, but I guess I should try again.

[withoutboats]

(NOT A CONTRIBUTION)

it doesn't seem like the straightforward syntax proposed here will need breaking revision in the future.

First, committing to the current behaviour means not having methods be send-by-default (opposite of the behaviour of async trait); maybe the team has already reached consensus on this, but it is a foreclosure on an option which is currently working well for the only actually used implementation of async traits. So it is a decision that is being made, not merely punted.

Even still, I don't think this is the only question at play here. The reality is that punted decisions have a tendency to stay punted for longer than initially expected. Async/await has already suffered reputational damage from how long its been a known-incomplete MVP and having cliffs like this very cliff. The team should consider what will be the effect of punting this issue.

In particular, the Send issue will create a much more difficult cliff. Currently the limitation is "async methods are not allowed in traits," which sucks but is easy to understand and plan for. Without resolving this, you will be using async methods just fine until suddenly you need a Send bound and you can't. These methods could be coming from an external library you don't control; you may need to refactor all of your code or you may need to drop using that library if you don't have a solution for this issue.

In my use case yesterday, the spawn was necessary to fix a bug in the code; if I couldn't've introduced it, I think we would have had to restructure our entire code base to fix that bug, because we used async methods before realising we would need to spawn in a generic context. This is not a trivial problem to run into.

[dwrensha]

EDIT: this post was based on me minunderstanding async methods and was corrected by withoutboats below. (click to see the original post

Even though work-stealing executors are the most commonly used in Rust, there are a sizable number of users that use single-threaded or thread-per-core executors. They won't run into this problem, at least with Send.

In capnp-rpc we use single-threaded futures, and it looks like we run into the same problem, except with 'static instead of Send.

Here's a distillation of the problem:
https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=0a984f5b66ea0333f74fb1b52f88dac0

My impression is that we're going to need a way to specify bounds on the returned futures of async trait methods, like:

trait MyServer {
    async (: 'static) fn foo(&mut self) -> Result<(),String>;
    async (: 'static) fn bar(&mut self) -> Result<(),String>;
}

I remember at some point in the past (like a few years ago) someone suggested that Send bounds should always be implicitly added. I think that's the wrong solution; it would prevent capnp-rpc from using async trait methods.

[withoutboats]

(NOT A CONTRIBUTION)

@dwrensha That doesn't work because those methods are not 'static - they borrow from self. A future will be 'static if all of the input arguments to its method are 'static, which there is syntax to express (e.g. if those methods took self by value, T: MyServer + 'static would be sufficient). it doesn't introduce the same problem as Send.

[dwrensha]

Ah, I see. I would have the same problem even if the async methods in question were not part of a trait.

more details (probably not relevant to the present issue)

We use this pattern, where a method has ephemeral access to &self and then returns a 'static future:

use std::future::Future;
use std::pin::Pin;

struct Foo {
    x: String
}

trait FooServer {
   fn foo(&self) -> Pin<Box<dyn Future<Output=()> + 'static>>;
}

impl FooServer for Foo {
    fn foo(&self) -> Pin<Box<dyn Future<Output=()> + 'static>> {
        // do stuff here that accesses &self.
        println!("{}", self.x);
        Box::pin(async {
            // no longer have access to &self.
            ()
        })
    }
}

pub fn main() {
    let foo  = Foo { x: "hi".into() };
    let fut : Pin<Box<dyn Future<Output=()> + 'static>> = foo.foo();
}

It would be nice if we could make foo() be an async method. I had mistakenly thought that the main obstacle was just that Rust does not allow async trait methods, but really the main obstacle is that I can't even write foo() as an async non-trait method, because as soon as it's an async method, the lifetime on self matches the lifetime on the returned future. I have no specific complaints that async methods work that way -- I had just forgotten about it.