Sudden failure to link statically on Arch Linux

[sanmai-NL]

It's always worked before. Turning back to compiling Rust after some period of other activities, and found out my binaries don't work anymore.

cargo new --bin bla

     Created binary (application) `bla` project

cd /tmp/bla/

env RUSTFLAGS="-C target-feature=+crt-static" cargo build --target x86_64-unknown-linux-musl

   Compiling bla v0.1.0 (file:///tmp/bla)
    Finished dev [unoptimized + debuginfo] target(s) in 0.35 secs

Actual

file ./target/x86_64-unknown-linux-musl/debug/bla

./target/x86_64-unknown-linux-musl/debug/bla: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=8c6e0c77c39c031c76d95379bfe2593165b13a70, with debug_info, not stripped

Expected

A description of an executable statically linked to musl libc.

Meta

rustc 1.21.0-nightly (1d2a6df38 2017-08-03)

OS

Arch Linux with musl package version 1.1.16-2.

[sanmai-NL]

Downgrading musl package to version 1.1.16-1 does not help. Any other options for testing?

[aidanhs]

Can you paste the output of ldd on the binary as well?

[sanmai-NL]

env -u LD_PRELOAD ldd -v ./target/x86_64-unknown-linux-musl/debug/bla

        statically linked

That seems fine then. The file output is normally correct, however, see e.g. to the bottom of this blog post.

My issue came up when I was trying to run the statically linked executables under Alpine Linux, as before. See this weird situation:

sudo env UNIFIED_CGROUP_HIERARCHY=no \
    rkt run --insecure-options=image docker://alpine:3.5 \
        --volume=bla,kind=host,source=/tmp/bla/target/x86_64-unknown-linux-musl/debug/ \
        --mount volume=bla,target=/bla/ \
        --exec=/bin/sh -- -c 'ldd -- /bla/bla; stat /bla/bla; /bla/bla; stat /lib64/ld-linux-x86-64.so.2'

[114845.186083] alpine[5]:      /lib64/ld-linux-x86-64.so.2 (0x55a8e0d15000)
[114845.186864] alpine[5]:   File: /bla/bla
[114845.187294] alpine[5]:   Size: 3912560      Blocks: 7648       IO Block: 4096   regular file
[114845.187433] alpine[5]: Device: 26h/38d      Inode: 3618974     Links: 2
[114845.187514] alpine[5]: Access: (0755/-rwxr-xr-x)  Uid: ( 1000/ UNKNOWN)   Gid: ( 1000/ UNKNOWN)
[114845.187583] alpine[5]: Access: 2017-08-04 20:28:41.000000000
[114845.187651] alpine[5]: Modify: 2017-08-04 18:23:00.000000000
[114845.187721] alpine[5]: Change: 2017-08-04 20:19:38.000000000
[114845.187793] alpine[5]: /bin/sh: /bla/bla: not found
[114845.187963] alpine[5]: stat: can't stat '/lib64/ld-linux-x86-64.so.2': No such file or directory

Seems the interpreter path isn't correct.

[aidanhs]

I've just tried to follow your steps on the latest nightly and I can't reproduce this - file reports the binary as statically linked. I have a vague recollection that Arch does some custom packaging of rust - can you confirm whether you're using the binary from rustup, or from arch itself?

Your choice of the version of musl package shouldn't make a difference (unless it's related to the custom packaging I mention) because Rust bundles the musl libc.a with the musl target.

[sanmai-NL]

Heh, it appears I had rustup installed as an Arch Linux package and via the common installer. Also, cargo now is included in the rustup Arch Linux package. I rectified the situation to only use the Arch Linux packaged rustup. I definitely use the x86_64-unknown-linux-musl target with the nightly-x86_64-unknown-linux-gnu toolchain. Nevertheless, the statically linked binaries still aren't being produced properly.

[sfackler]

I can reproduce this as well with rustup-managed 1.19 on Arch, but not on Debian Jessie, where the file type is

target/x86_64-unknown-linux-musl/debug/foo: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, not stripped

It seems like it'd probably have to do with the way Arch's musl toolchain is configured?

[sfackler]

Though it looks like you haven't told rustc to use musl-gcc as the linker, since that makes the resulting binary use /lib/ld-musl-x86_64.so.1 as its interpreter rather than /lib64/ld-linux-x86-64.so.2.

[aidanhs]

@sfackler unless I've misunderstood the intent, linking a static binary with the musl target should not require a custom linker unless this is an oddity of arch. In fact, the host musl toolchain should be totally unused (unless depending on a crate that invokes musl-gcc as part of a build script).

[sfackler]

Ah! There is a difference between Debian and Arch's musl-gcc wrappers - Debian's pulls in spec files that disable PIE in addition to the normal MUSL one.

This custom linker setup produces fully static binaries on Arch:

test-gcc

#!/bin/sh                                                                                                                                                                                                                                              
exec "${REALGCC:-gcc}" "$@" -specs "$HOME/no-pie-compile.specs" -specs "$HOME/no-pie-link.specs"

no-pie-link.specs

*self_spec:                                                                                                                                                                                                                                            
+ %{!shared:%{!r:%{!fPIE:%{!pie:-fno-PIE -no-pie}}}}

*self_spec:                                                                                                                                                                                                                                            
+ %{!r:%{!fpie:%{!fPIE:%{!fpic:%{!fPIC:%{!fno-pic:-fno-PIE}}}}}}

It looks like we may want to disable PIE for MUSL targets.

[sfackler]

We already appear to not enable PIE for MUSL targets, but it looks like we should actually explicitly be disabling PIE rather than relying on the linker default.

rust/src/librustc_back/target/linux_musl_base.rs

Line 68 in cd72f2e

base.position_independent_executables = false;

[sfackler]

It seems like #40113 may resolve this.