Minimal path to stabilization

[alexcrichton]

I wanted to open a sort of tracking issue for getting stdsimd into the standard library and on the path to stable Rust. That's sort of what we've wanted all along! What I propose here may seem a bit radical, but hear me out!

First and foremost I see the purpose of the stdsimd crate to be providing definitions of the vendor intrinsics. These functions are all unsafe as previously discussed and use #[target_feature] to implement themselves. Vendors (like Intel) define how many functions are here and what their names are. The important thing about these intrinsics is that how they're defined is compiler dependent. This, in turn, forces them to be implemented with unstable functionality, namely we must move this to the standard library in one form or another to be stable.

Next up if we have vendor intrinsics we also have the types! Each vendor intrinsic has types associated with it, although we are thinking of differing from the upstream vendors with more "principled" types like u8x16 instead of __m128i. As a result these types are also defined in the standard library.

After this, however, I'd propose we stop. This issue is all about the minimal path to stabilization (while still being useful) and I believe that type definitions plus intrinsics is sufficient. I say this namely because that's all C/C++ have, type definitions and intrinsics. I personally very much want to stabilize more convenient names like f32x4::splat or u16x8::extract where I don't have to remember janky names from Intel, but this is always backwards compatible to add at a later date. I think we have a lot more to gain from a little bit of SIMD now rather than "super portable SIMD right out the gate" much later. I hope to soon after the first pass of stabilization start looking to stabilizing the more convenient and powerful APIs (like Add for u32x8).

Once we have this then I think we also need to stabilize the #[target_feature] attribute and its associated features. This namely involves finishing implementing RFC 2045, a slight tweak to #[target_feature] syntax.

Up to this point I believe the last remaining issue is "what happens on the classical ABI mismatch issue". Namely, what does rustc do with this function:

#[target_feature = "+avx"]
fn foo(a: u64x8) -> u64x8 {
    a
}

unsafe fn bar() {
    foo(mem::zeroed());
}

(or something like that)

Notably here the bar function does not have the avx feature enabled, so the ABI it will invoke foo with differs than that of the ABI that foo is expecting, hence causing a mismatch. I would propose a different strategy from RFC 2045 talked about at Rust Belt Rust with some others, namely inserting shims. This I believe solves the function pointer problem as well! Namely:

• If you invoke a function with a different set of target features and you have an argument that's ABI-relevant, the compiler inserts a shim which "fixes" the invocation. Namely it would fix the above to:

#[target_feature = "+avx"]
fn foo(a: u64x8) -> u64x8 {
    a
}

#[target_feature = "+avx"]
fn foo_shim(a: &u64x8, out: &mut u64x8) {
    *out = foo(*a);
}

unsafe fn bar() {
    foo_shim(mem::zeroed());
}

• All function pointers have the "default" ABI in that they have no target features enabled. This means that if you coerced foo to a function pointer above you would actually get a foo_shim function pointer, regardless of where the function pointer is created.

When this is all assembled I believe we've got a very short path to stable SIMD on Rust today. This path should be sound so long as you have the right CPU feature checks in place, extensible to add more portable extension in the future, and complete in that we have accomodated for the existence of all vendor intrinsics.

So in summary, I believe we have three blockers for stabilization:

• Agree on the minimal API consisting purely of unsafe vendor intrinsics and type definitions, no trait impls or other methods on the type definitions stabilized yet (they can and will be added later!)
• Implement the minor tweaks to the #[target_feature] attribute as specified in RFC 2045
• Implement generation of "shim" functions in trans to work around the "ABI mismatch" problem

Once this is all done and agreed on I think we can also look to moving this crate into the standard library. I believe that would entail basically adding a src/stdsimd submodule pointing to this repository, adding a pub mod simd to src/libcore/lib.rs which points to this crate's root (probably setting some sort of #[cfg] along the way so this crate knows what's happening), and then finally adding a bunch of stability annotations all throughout the crate.

---

Ok so that may be a bit much, but I'm curious what others think!

cc @gnzlbg
cc @BurntSushi
cc @nikomatsakis
cc @withouboats
cc @hdevalence

[aturon]

cc @withoutboats (typo in issue)

[gnzlbg]

I agree 100% with everything you said.

I'd like to add one thing:

I think we have a lot more to gain from a little bit of SIMD now rather than "super portable SIMD right out the gate" much later.

I think we should prototype a "super portable SIMD library" that uses stdsimd for the intrinsics before we stabilize anything. Nothing fancy, just some vector arithmetic, sqrt, and not much more, just to check whether it can be built on top of stdsimd or not, and if not, what the problems are.

I've started already to do this and will commit it as an example/ (or if it gets too big, as its own crate as soon as we do the next release of stdsimd), but I think doing this in parallel will either give us confidence about the design or warning signs about things we might have to tune.

Just to be clear: I think a portable SIMD library does not belong in std (there are many ways to do that, with different trade-offs), but I think we should make sure that whatever we stabilize is at least on the right path to allow building those libraries on stable Rust some day.

And I have one nitpick:

I would propose a different strategy from RFC 2045 talked about at Rust Belt Rust with some others, namely inserting shims.

I think that inserting shims is a great idea! The shims are a zero-cost abstraction, that is, if you want to do this there is no more efficient way of doing it. However, if I ever want to avoid the cost of the shims, I'd like to have enough control to do that.

So I would prefer if the compiler would hard error when this happens (so that I know where these cases are), and we would provide some functionality to manually tell the compiler "insert shims here". We can always make it automatic in the future without breaking backward compatibility.

Anyhow, I don't want to derail this thread with the nitpick, we can discuss that somewhere else.

[withoutboats]

@alexcrichton This plan sounds solid to me. 👍

In terms of ordering work, does anything block putting the intrinsics into libcore behind a feature flag right now? While all of these items need to be done before stabilizing anything, I think they could be done in any order.

@gnzlbg

I think we should make sure that whatever we stabilize is at least on the right path to allow building those libraries on stable Rust some day.

Since we're talking about just exposing the lowest-level platform intrinsics, I don't see a lot of choices between what paths we're on. This proposal seems like stabilizing just the things there are no alternatives between.

Do you have concrete examples of things you think are uncertain now and that you don't want to lock down through stabilization?

However, if I ever want to avoid the cost of the shims, I'd like to have enough control to do that.

Isn't this supported by using the cfg based system (which I think is fully implemented and stable today) instead of the runtime system? That is, you cfg out the functions unless the target supports these features without any shimming.

[gnzlbg]

@withoutboats check out the examples/nbody.rs which currently uses static feature detection. We would like to improve that to select the best instruction at run-time. We can already do that today, but it is a bit unclear whether how that looks is what we want.

For example, a safe SIMD library can do the runtime detection on each intrinsic call, but that's basically adding an atomic load to each simd operation in the library. Ideally, the dispatch would happen at a higher level (maybe even at the level of fn main()). It is unclear how to write a portable SIMD library that would support doing this at this point.

Isn't this supported by using the cfg based system (which I think is fully implemented and stable today) instead of the runtime system? That is, you cfg out the functions unless the target supports these features without any shimming.

By cost, I meant the cost of converting e.g. 2x 128bit-wide registers into a 256 bit one and vice-versa. The shims do this conversion, which is not free. This conversion can be done manually today, e.g., by manually writing the shims. The question is whether we want to induce this cost "silently" or whether we want instead offer a way to automatically generate these shims, but such that the user has to say that it wants them. I'd rather start conservatively, but either way we are moving in the right direction. Once the shim generation is implemented we can revise whether we want to generate them automatically, or error and let the user decide (ideally in such a way that a library that does it automatically can be built on top of it).

---

FWIW, this means that we can stabilize the intrinsics, the vector types, and #[target_feature] but that maybe we should probably explore writing high-level SIMD wrappers a bit before stabilizing cfg(target_feature) and cfg_feature_enabled.

[alexcrichton]

@withoutboats correct yeah, I think we could make inroads immediately on including this library into libcore.

@gnzlbg

I'd agree that we should play around with this interface some more! That's sort of what we expect to naturally happen as this becomes more broadly available on nightly (e.g. included into libcore). I'm a little skeptical like @withoutboats, however, that this will turn up a whole lot and I wouldn't want to necessarily block on a proof-of-concept library (depending on the circumstances)

In my mind the shims are basically necessary for this to be a "reasonable" feature in terms of usability and expectations of how Rust code works "in the large". I'd imagine that the impact of such shims would show up very rarely in practice.

[gnzlbg]

Yes, I don't think we should block moving this into core on a prototype of a higher level library. From core to Rust stable there is more than enough time to do that.

I'd imagine that the impact of such shims would show up very rarely in practice

Yes, I also expect that these shims will be inserted very rarely in practice, and when it does, what most people will want is the shims to be generated anyways.

[BurntSushi]

@alexcrichton Thanks for this proposal! Broadly speaking, I'm OK with this. I would however like to note some costs so that we're all on the same page.

In particular, if we omit everything except the type definitions themselves, then I think the following is true:

• Building a portable SIMD library (outside of std, without LLVM's help) will be hard. @stoklund talks about this here: https://internals.rust-lang.org/t/getting-explicit-simd-on-stable-rust/4380/209 --- That in turn means that the current simd library won't have an obvious path to stabilization. (This last point isn't terribly concerning, since it's not clear what role the simd crate will play anyway. But still, if someone wanted to go out and write their own Add impls, for example, then it's not trivial.)
• Using many of the intrinsics will be even more painful, since casting between vector types feels like a pretty common thing to do. Without the various as methods or From impls that we have today, I think folks will be forced to transmute or pointer cast. I think @sfackler had some thoughts on this.

I do think this is a fine first step though!

[sfackler]

Yeah - I was porting StreamVByte from 32 to 64 bit values. The encode kernel interprets its vectors as all of the u8x32, u32x8, i32x8, and u64x4 types. I have to convert at least one of the vector arguments to every single intrinsic call.

I don't know how common that stye of SIMD workload is, but the type safety was a net negative. If the Into impls went away it'd be even worse.

[alexcrichton]

@BurntSushi excellent points! I'm just trying to thread the needle as "quickly as possible" in the sense of building up a conservative base that can be experimented with while ensuring we can continue to add features. I'd hope enabling the simd crate as-is today would be the next stop of stabilization for SIMD in Rust!

@sfackler an interesting case study! Did the original code rely on implicit conversions done by the C compiler? I think we're slighly more principled than __m256i for example (having a bunch of widths other than that and typing the intrinsics more strictly than the Intel documentation says). In that case we're definitely going to be more painful than with C, but if the C intrinsics have different types themselves then it's sort of like porting line-for-line arithmetic in C to Rust, it just likely needs more casts.

[gnzlbg]

So FWIW I wanted to reimplement simd_cast on top of stdsimd because I have the same problem as @sflacker and without the proper as methods nd From trait impls this is a pain to do because 1) there are not intrinsic for al conversions and 2) because to offer portability I need to use llvm sind cast any ways. Adding enough as methods and from impls should fix this so I am hopeful that this will not be a major issue to overcome with the current approach.

…

On Mon 30. Oct 2017 at 23:21, Steven Fackler ***@***.***> wrote: Yeah - I was porting StreamVByte <https://github.com/lemire/streamvbyte> from 32 to 64 bit values. The encode kernel <https://github.com/sfackler/stream-vbyte64/blob/ea0d5b0afdf97f31473fafbf33d460fbbb313785/src/lib.rs#L115> interprets its vectors as all of the u8x32, u32x8, i32x8, and u64x4 types. I have to convert at least one of the vector arguments to every single intrinsic call. I don't know how common that stye of SIMD workload is, but the type safety was a net negative. If the Into impls went away it'd be even worse. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#159 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA3NptJ5zoPgiIvZD1suFB5DblmUG1jdks5sxkv3gaJpZM4QJyau> .

[sfackler]

@alexcrichton From what I can tell, the C code is able to work entirely with _m128is, with a union to type pun bytes and ints out: https://github.com/lemire/streamvbyte/blob/master/src/streamvbyte.c#L183.

[hdevalence]

My understanding is that many of the From casts currently in the stdsimd crate are already effectively transmutes/bitcasts -- for instance, i32x8 vs i64x4. Is this correct? Does this mean that the question isn't whether people will transmute or not, but just whether they'll do it using stdsimd-provided impls?

[gnzlbg]

There are two things people want/need: - 1) element-wise as cast: f32x4 as f64x4 - 2) same vector width transmutes: u32x4 as u64x2 We currently offer the as casts to do both, plus the from implementations. I think we should be a more principled here, but this is something that we'll definitely nail down before stabilization.

…

On Tue 31. Oct 2017 at 00:32, Henry de Valence ***@***.***> wrote: My understanding is that many of the From casts currently in the stdsimd crate are already effectively transmutes/bitcasts -- for instance, i32x8 vs i64x4. Is this correct? Does this mean that the question isn't whether people will transmute or not, but just whether they'll do it using stdsimd-provided impls? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#159 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA3NppKUVFaZldmtxv6Cdc15UArt1_xJks5sxlyZgaJpZM4QJyau> .

[alexcrichton]

In general I just wanted to make a proposal which had stabilization as an option in the near future. We can of course add more requirements to that, for example auditing the current API of u32x8, casts and trait impls and all. I fear that will delay stabilization further, however, and at this point I feel we have more to benefit from getting something out there rather than being perma-unstable.

[hdevalence]

@gnzlbg I'm not sure I understand, what do you mean by more principled?