rust libs are not reproducible from 1.75

[sundeep-kokkonda]

Hello,

We're building rust in Yocto framework for a custom target, and there we are facing a reproducible issue with the libs generated during the build. When the rust is build in two different paths the generated libs are not identical, those are getting differed in .rustc section. This is observed from the rust 1.75.0 onwards.
(We've enabled remap-debuginfo option but still this issue is happening, this option was learned from issue discussion https://internals.rust-lang.org/t/librustc-driver-so-not-reproducible/19639 and this was working fine till 1.74.1)

I've analyzed the changes from 1.74.1 to 1.75.0, there are changes related to Cargo, llvm & some changes w.r.t to bolt feature/optimization are there. I've tried a few builds by reverting these changes but still the issue is reproducing.

There are a few commits in 85c0ce2 for src/bootstrap/src/core/builder.rs & src/bootstrap/src/bin/rustc.rs files which have changes related to remap-debuginfo option and I suspect this commit might causing the issue. I tried build by reverting the commit & with the latest changes in 4b7e0a0 but still issue is reproduing. There is a new RUSTC_CARGO_REGISTRY_SRC_TO_REMAP is added in this commit. I added a few debug statements in here to see the build paths passed, I can see those are passed correctly to RUSTC_CARGO_REGISTRY_SRC_TO_REMAP.

Any hints on what might cause reproducible issues with this commit on rust 1.75? Let me know if any other info is needed.

Non-reproducible libs are here:

[bjorn3]

Could you see if you can extract both versions of the .rustc section, follow the instructions in https://internals.rust-lang.org/t/librustc-driver-so-not-reproducible/19639/10 to decompress and then show a diff of the decompressed data?

Edit: Forgot that I removed metadata compression for dylibs a while ago. So you can just diff the .rustc section directly.

[saethlin]

I'm assigning myself because I'm going to try to build a tool to help with this. It might be pretty cooked.

[sundeep-kokkonda]

Could you see if you can extract both versions of the .rustc section, follow the instructions in https://internals.rust-lang.org/t/librustc-driver-so-not-reproducible/19639/10 to decompress and then show a diff of the decompressed data?

Edit: Forgot that I removed metadata compression for dylibs a while ago. So you can just diff the .rustc section directly.

Attached librustc_driver-cbccc190731a002e & libcstr-13225ad2e5d1b622 libs .rustc section diff here (There are many more libs differed).

Diff for .rustc section of librustc_driver-cbccc190731a002e (in librustc_driver the diff is huge, below image is part of it. The attached librustc_driver_report.pdf is full diff report)

librustc_driver_report.pdf

Diff for .rustc section of libcstr-13225ad2e5d1b622

[sundeep-kokkonda]

I'm assigning myself because I'm going to try to build a tool to help with this. It might be pretty cooked.

@saethlin Tool for what?

[saethlin]

Figuring out what these bytes are.

[bjorn3]

Looks like some paths aren't getting remapped as they should.

[sundeep-kokkonda]

Looks like some paths aren't getting remapped as they should.

Is there a way to pull out that info, which paths, build stage, crate info etc? The verbose build log shows me the --remap-path-prefix option is passed during build of each crate and the debug statements (I added) in 4b7e0a0 also shows correct paths.

[sundeep-kokkonda]

Edit: Forgot that I removed metadata compression for dylibs a while ago. So you can just diff the .rustc section directly.

Does this mean the -Zls options mentioned here are not supporting now?