Sendable conformance on _AsyncBytesBuffer.Storage allows unsafe concurrent access

[mhjacobson]

In AsyncBufferedByteIterator.swift, _AsyncBytesBuffer.Storage (a class) conforms to Sendable, allowing it to be passed across concurrency boundaries. Since concurrent access is allowed, classes conforming to Sendable are responsible for implementing internal synchronization, but Storage doesn't have any.

This means that it's possible to construct data races on the buffer of an _AsyncBytesBuffer.Storage (through copies of an AsyncBufferedByteIterator). (See this gist for an example.) Here's what happens:

1. Multiple instances of _AsyncBytesBuffer value can share a single Storage. If the multiple instances are read from on separate threads, then their readFunctions may be called concurrently, with the same pointer argument. Since the job of the readFunction is to fill in the pointee bytes, there is a write/write data race on those bytes

2. Similarly, one copy may be reading from the Storage while another is calling its readFunction, producing a read/write data race on the bytes

[mhjacobson]

I'll note that, disregarding the data races, even non-concurrent use of multiple copies of an AsyncBufferedBytesIterator has maybe unexpected behavior. If one copy reloads before other copies have reached the end of the buffer, the other copies will skip bytes.

Therefore, it's not clear to me what guarantees AsyncBufferedBytesIterator intends to make about use of copies in general.

However, I might suggest that if using multiple copies of an AsyncBufferedBytesIterator is discouraged, then perhaps AsyncBufferedBytesIterator could itself be made a reference type (at least, until the language supports move-only types).

One solution might be to make it an actor, which would fix the data races, too...

[phausler]

actor in this case is way too expensive and will introduce another race to an out of order access.