LLVM can't support partial register clobbers

[jstarks]

On x86_64-pc-windows-msvc, the system ABI considers xmm0-5 as volatile/caller-save registers, and xmm6-15 as non-volatile/callee-save registers. However, clobber_abi("C") incorrectly marks xmm6-15 as clobbered, which produces a bunch of wasteful register saves/restores.

This behaves according to the documentation (which specifies that xmm6-15 will be clobbered), but it's almost certainly not what the developer wanted due to the unnecessary clobbering causing performance problems. The workaround is to avoid using clobber_abi on Windows and to manually clobber registers; this is a pain and may cause problems down the line as more volatile registers are added to the ABI.

So a breaking change seems warranted here.

Godbolt link

#[no_mangle]
pub fn foo() {
    unsafe {
        std::arch::asm! {
            "call bar",
            clobber_abi("C"),
        }
    }
}

Asm:

foo:
        sub     rsp, 168
        movaps  xmmword ptr [rsp + 144], xmm15
        movaps  xmmword ptr [rsp + 128], xmm14
        movaps  xmmword ptr [rsp + 112], xmm13
        movaps  xmmword ptr [rsp + 96], xmm12
        movaps  xmmword ptr [rsp + 80], xmm11
        movaps  xmmword ptr [rsp + 64], xmm10
        movaps  xmmword ptr [rsp + 48], xmm9
        movaps  xmmword ptr [rsp + 32], xmm8
        movaps  xmmword ptr [rsp + 16], xmm7
        movaps  xmmword ptr [rsp], xmm6

        call    bar

        movaps  xmm6, xmmword ptr [rsp]
        movaps  xmm7, xmmword ptr [rsp + 16]
        movaps  xmm8, xmmword ptr [rsp + 32]
        movaps  xmm9, xmmword ptr [rsp + 48]
        movaps  xmm10, xmmword ptr [rsp + 64]
        movaps  xmm11, xmmword ptr [rsp + 80]
        movaps  xmm12, xmmword ptr [rsp + 96]
        movaps  xmm13, xmmword ptr [rsp + 112]
        movaps  xmm14, xmmword ptr [rsp + 128]
        movaps  xmm15, xmmword ptr [rsp + 144]
        add     rsp, 168
        ret

I expected that foo would just establish a frame and call bar without saving/restoring of the XMM registers.

Meta

rustc --version --verbose:

rustc 1.78.0 (9b00956e5 2024-04-29)
binary: rustc
commit-hash: 9b00956e56009bab2aa15d7bff10916599e3d6d6
commit-date: 2024-04-29
host: x86_64-unknown-linux-gnu
release: 1.78.0
LLVM version: 18.1.2

But this repros in nightly.

[workingjubilee]

Thank you for pointing this out!

It feels dubious to call it a "breaking change" to apply this diff:

 /// Always returns true.
 pub fn returns_true() -> bool {
+    true
-    false
 }

But regardless of whether it is or isn't, I agree that clobber_abi should definitely do what it says it does, because it is very important that unsafe code uphold its contracts exactly. We should definitely fix this soon, considering that we're about to see x86 architectures with 32 GPRs!

[jstarks]

It looks like this was known by the author (@Amanieu):

            InlineAsmClobberAbi::X86_64Win => clobbered_regs! {
                X86 X86InlineAsmReg {
                    // rdi and rsi are callee-saved on windows
                    ax, cx, dx, r8, r9, r10, r11,

                    // xmm6-xmm15 are callee-saved on windows, but we need to
                    // mark them as clobbered anyways because the upper portions
                    // of ymm6-ymm15 are volatile.
                    xmm0, xmm1, xmm2, xmm3, xmm4, xmm5, xmm6, xmm7,
                    xmm8, xmm9, xmm10, xmm11, xmm12, xmm13, xmm14, xmm15,
                    zmm16, zmm17, zmm18, zmm19, zmm20, zmm21, zmm22, zmm23,
                    zmm24, zmm25, zmm26, zmm27, zmm28, zmm29, zmm30, zmm31,

                    k0, k1, k2, k3, k4, k5, k6, k7,

                    mm0, mm1, mm2, mm3, mm4, mm5, mm6, mm7,
                    st0, st1, st2, st3, st4, st5, st6, st7,
                    tmm0, tmm1, tmm2, tmm3, tmm4, tmm5, tmm6, tmm7,
                }
            },

[workingjubilee]

Oh, right!

I don't think we have a reasonable way to mark "only half a register" as clobbered!

[workingjubilee]

Does LLVM have a way to express these partial clobbers in its assembly language?

[Amanieu]

And if you look just below, you'll see we have the same issue on AArch64.

                AArch64 AArch64InlineAsmReg {
                    x0, x1, x2, x3, x4, x5, x6, x7,
                    x8, x9, x10, x11, x12, x13, x14, x15,
                    x16, x17, x18, x30,

                    // Technically the low 64 bits of v8-v15 are preserved, but
                    // we have no way of expressing this using clobbers.
                    v0, v1, v2, v3, v4, v5, v6, v7,
                    v8, v9, v10, v11, v12, v13, v14, v15,
                    v16, v17, v18, v19, v20, v21, v22, v23,
                    v24, v25, v26, v27, v28, v29, v30, v31,

                    p0, p1, p2, p3, p4, p5, p6, p7,
                    p8, p9, p10, p11, p12, p13, p14, p15,
                    ffr,

                }

This is unfortunately a limitation of LLVM: it doesn't give us a way to indicate only half of a register is clobbered.

[workingjubilee]

Yeah, ideally there would be a special "vzeroupper" clobber but there isn't one, it seems.