Code coverage does not work on armv7-unknown-linux-gnueabihf due to missing `profiler_builtins`

[FibreFoX]

I am new to the Rust ecosystem, but as part of my own learning curve, I wanted to create a small personal project which includes test coverage executed on my raspberry pi server.

My setup contains a self-hosted Gitlab-instance (which runs inside docker) running on a 32-bit based Raspbian on my Raspberry Pi 4 (with 8gb RAM). After some hours of trying to get grcov working I tried to reproduce my problems, and found out, that profiler_builtins are missing for armv7-unknown-linux-gnueabihf.

I opened up an issue on the GRCOV-project (mozilla/grcov#508), because I first thought I was missing some important instruction to set up my system. It now seems to me that the package provided via rustup does not include the required crates/libs to make it work to have code coverage being analysed.

Steps to reproduce

• install the docker engine + docker cli (locally on intel/amd-based system; and on a raspberry pi with 32bit operating system)
• create a new project in a new folder rust-project containing a normal project (inited via cargo init)

root@9c75a90fc396:/# tree ./rust-project
./rust-project
|-- Cargo.lock
|-- Cargo.toml
`-- src
    `-- main.rs

Content of main.rs:

fn main() {
	println!("Message: {}", get_message());
}

fn get_message() -> &'static str {
	return "Hello, world!";
}

#[cfg(test)]
mod tests {
	use super::get_message;
	#[test]
	fn check_message() {
		assert_eq!(get_message(), "Hello, world!");
	}
}

• create a Dockerfile aside of that folder:

# Dockerfile
FROM rust:latest

ARG DEBIAN_FRONTEND=noninteractive

RUN apt-get update && apt-get install -y lcov
RUN rustup toolchain install nightly
RUN cargo install grcov

COPY ./rust-project /rust-project

WORKDIR /rust-project

• create that container: docker build -t rust-grcov-raspberry-bug:latest .
• run container: docker run --rm -ti rust-grcov-raspberry-bug:latest
• execute the following inside the container:

export CARGO_INCREMENTAL=0
export RUSTFLAGS="-Zprofile -Ccodegen-units=1 -Copt-level=0 -Clink-dead-code -Coverflow-checks=off -Zpanic_abort_tests -Cpanic=abort"
export RUSTDOCFLAGS="-Cpanic=abort"
cargo +nightly test --verbose

Result on Raspberry Pi 4 (32bit arm)

root@bfbc64ce82c2:/rust-project# cargo +nightly test --verbose
   Compiling rust-project v0.1.0 (/rust-project)
     Running `rustc --crate-name rust_project --edition=2018 src/main.rs --error-format=json --json=diagnostic-rendered-ansi --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 --test -C metadata=a223de0c0bfa7c21 -C extra-filename=-a223de0c0bfa7c21 --out-dir /rust-project/target/debug/deps -L dependency=/rust-project/target/debug/deps -Zprofile -Ccodegen-units=1 -Copt-level=0 -Clink-dead-code -Coverflow-checks=off -Zpanic_abort_tests -Cpanic=abort`
error[E0463]: can't find crate for `profiler_builtins`
  |
  = note: the compiler may have been built without the profiler runtime

error: aborting due to previous error

For more information about this error, try `rustc --explain E0463`.
error: could not compile `rust-project`

Caused by:
  process didn't exit successfully: `rustc --crate-name rust_project --edition=2018 src/main.rs --error-format=json --json=diagnostic-rendered-ansi --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 --test -C metadata=a223de0c0bfa7c21 -C extra-filename=-a223de0c0bfa7c21 --out-dir /rust-project/target/debug/deps -L dependency=/rust-project/target/debug/deps -Zprofile -Ccodegen-units=1 -Copt-level=0 -Clink-dead-code -Coverflow-checks=off -Zpanic_abort_tests -Cpanic=abort` (exit code: 1)

root@deea46aaee17:/rust-project# rustc --version --verbose
rustc 1.47.0 (18bf6b4f0 2020-10-07)
binary: rustc
commit-hash: 18bf6b4f01a6feaf7259ba7cdae58031af1b7b39
commit-date: 2020-10-07
host: armv7-unknown-linux-gnueabihf
release: 1.47.0
LLVM version: 11.0

root@a0589d6db0a9:/rust-project# rustc +nightly --version --verbose
rustc 1.49.0-nightly (b2d115f6d 2020-11-07)
binary: rustc
commit-hash: b2d115f6db5172c961dfeb50de15f35784dbc7c9
commit-date: 2020-11-07
host: armv7-unknown-linux-gnueabihf
release: 1.49.0-nightly

Result on local Windows 10 using docker-desktop (64bit amd64)

root@9c75a90fc396:/rust-project# cargo +nightly test --verbose
   Compiling rust-project v0.1.0 (/rust-project)
     Running `rustc --crate-name rust_project --edition=2018 src/main.rs --error-format=json --json=diagnostic-rendered-ansi --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 --test -C metadata=e2970e522a94ba3c -C extra-filename=-e2970e522a94ba3c --out-dir /rust-project/target/debug/deps -L dependency=/rust-project/target/debug/deps -Zprofile -Ccodegen-units=1 -Copt-level=0 -Clink-dead-code -Coverflow-checks=off -Zpanic_abort_tests -Cpanic=abort`
    Finished test [unoptimized + debuginfo] target(s) in 0.44s
     Running `/rust-project/target/debug/deps/rust_project-e2970e522a94ba3c`

running 1 test
test tests::check_message ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

root@93adfcd192a9:/rust-project# rustc --version --verbose
rustc 1.47.0 (18bf6b4f0 2020-10-07)
binary: rustc
commit-hash: 18bf6b4f01a6feaf7259ba7cdae58031af1b7b39
commit-date: 2020-10-07
host: x86_64-unknown-linux-gnu
release: 1.47.0
LLVM version: 11.0

root@93adfcd192a9:/rust-project# rustc +nightly --version --verbose
rustc 1.49.0-nightly (b2d115f6d 2020-11-07)
binary: rustc
commit-hash: b2d115f6db5172c961dfeb50de15f35784dbc7c9
commit-date: 2020-11-07
host: x86_64-unknown-linux-gnu
release: 1.49.0-nightly

[richkadel]

My guess is the version of rustc you are compiling with was built without profiler = true set in its build configuration (e.g., config.toml).

[richkadel]

This may be related to #79556

[FibreFoX]

Hm, #79556 is related to musl, you sure it is related? As I am using official rust:latest-image, where can I check/see, if it was built with profiler = true ? The defaults seem to have no profiler to get built into

rust/config.toml.example

Line 268 in 2225ee1

#profiler = false

[richkadel]

Related in the sense that both issues report that the build of the compiler for their target platforms does not set "profiler = true".

Yes, the default is "profiler = false", but various pre-built distros enable "profiler = true". Some don't. I don't know who makes those decisions, whether we should change this to enable "profiler = true" by default in the future, and/or whether the profiler_builtins library works or not for some platforms.

[rich-g]

Is there any update on this? Is this simply a matter of adding --enable-profiler to the Dockerfile, or are there other pieces missing too?

[FibreFoX]

@rich-g I think this is something pretty low-level to be enabled first inside the compiled rust docker image, nothing that can just be activated via that parameter. I will try to re-check on the upcoming weekend to see if there has something changed silently.

[FibreFoX]

@rich-g Nope, still the same issue. Retried on a fresh install of the whole system (Bullseye-based raspbianOS ... whatever its called) and got the same output:

root@ae1e55809c68:/rust-project# cargo +nightly test --verbose
   Compiling rust-project v0.1.0 (/rust-project)
     Running `rustc --crate-name rust_project --edition=2018 src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 --test -C metadata=0d091f013ad5ec96 -C extra-filename=-0d091f013ad5ec96 --out-dir /rust-project/target/debug/deps -L dependency=/rust-project/target/debug/deps -Zprofile -Ccodegen-units=1 -Copt-level=0 -Clink-dead-code -Coverflow-checks=off -Zpanic_abort_tests -Cpanic=abort`
error[E0463]: can't find crate for `profiler_builtins`
  |
  = note: the compiler may have been built without the profiler runtime

For more information about this error, try `rustc --explain E0463`.
error: could not compile `rust-project` due to previous error

Caused by:
  process didn't exit successfully: `rustc --crate-name rust_project --edition=2018 src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 --test -C metadata=0d091f013ad5ec96 -C extra-filename=-0d091f013ad5ec96 --out-dir /rust-project/target/debug/deps -L dependency=/rust-project/target/debug/deps -Zprofile -Ccodegen-units=1 -Copt-level=0 -Clink-dead-code -Coverflow-checks=off -Zpanic_abort_tests -Cpanic=abort` (exit status: 1)
root@ae1e55809c68:/rust-project#

versions:

root@ae1e55809c68:/rust-project# rustc --version --verbose
rustc 1.60.0 (7737e0b5c 2022-04-04)
binary: rustc
commit-hash: 7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c
commit-date: 2022-04-04
host: armv7-unknown-linux-gnueabihf
release: 1.60.0
LLVM version: 14.0.0

root@ae1e55809c68:/rust-project# rustc +nightly --version --verbose
rustc 1.62.0-nightly (30f386087 2022-05-05)
binary: rustc
commit-hash: 30f386087564243ab88a93c984c265290a31580b
commit-date: 2022-05-05
host: armv7-unknown-linux-gnueabihf
release: 1.62.0-nightly
LLVM version: 14.0.1

EDIT: just to point out, the used rust comes from the docker-container, not the hosting OS

[clarfonthey]

Potential duplicate of #79556?

[FibreFoX]

@clarfonthey similar but no duplicate, because the other issue targets MUSL, this is ARMv7-specific 😸

[clarfonthey]

Right, but the root cause appears to be an issue with the releases provided for tier 2 targets, which affects both.

[FibreFoX]

This already got suggested here: #79640 (comment)

All it comes down to "profiler_builtins" not being present. If both issues are having the same root cause, thats okay, but as every target is configured differently (like already mentioned here: #79640 (comment)), so I do not see a duplicate here. #79556 is specificly targeting MUSL, this is non-MUSL and includes an easy reproducable example.