Tracking issue for MIR-only RLIBs

[michaelwoerister]

There's been some talk about switching RLIBs to "MIR-only", that is, make RLIBs contain only the MIR representation of a program and not the LLVM IR and machine code as they do now. This issue will try to collect some advantages, disadvantages, and other concerns such an approach would entail:

Advantages

• Less code duplication, which has four benefits:
  • RLIBs would be smaller because they would not contain LLVM IR and machine code anymore.
  • RLIBs and leaf crates would be smaller because, at the moment, instantiations of generic functions show up multiple times in the object code and LLVM IR.
  • RLIBs and leaf crates would be smaller because the compiler would be able instantiate monomorphic functions strictly on demand, as @japaric points out.
  • Possibly faster whole-project compiles, since generic instances are never compiled multiple times (although see "Disadvantages")
• RLIBs would compile faster because the trans and LLVM passes would always be skipped (much like when compiling with -C metadata).
• At the moment libstd is compiled with -Cdebuginfo=1, which is good in general but as a side-effect increases the size of Rust binaries, even if they are built without debuginfo (because the debuginfo from libstd gets statically linked into the binaries). This problem would not exist with MIR-only rlibs.
• In the past we've had problems with WeakODR linkage and COMDAT sections on MinGW. WeakODR linkage is one way to deal with duplicate generic instances and avoiding those would also remove any reason to use WeakODR.
• We would always get LTO-grade compiler optimizations since all code is available at codegen time.
• Some targets, like NVPTX, don't seem to support regular linking (see NVPTX: non-inlined functions can't be used cross crate #38787). Only generating object code in leaf crates would solve this problem.
• There seems be some indication that MIR-only RLIBs would help with making the Rust compiler more backend agnostic (see WASM-related issue Migrate wasm target to LLVM wasm backend #38804).
• Generating LLVM IR only in leaf crates would make it easier to add comprehensive LLVM-based instrumentation like LeakSanitizer without recompiling libstd (see LeakSanitizer, ThreadSanitizer, AddressSanitizer and MemorySanitizer support #38699), as @japaric points out.
• All Rust code (even that from libstd) can be compiled with -C target-cpu=native, potentially resulting in better code, as @japaric points out.
• The build process of multi-crate project would gain more parallelism, since downstream crates don't need to wait for upstream crate's codegen, even though they could already compile up until the linking phase, as @est31 points out.

Disadvantages

• The leaf crates (executables, staticlibs, dylibs, cdylibs) would take more time to compile because
  1. the machine code of monomorphic functions from upstream crates would not be "cached" anymore, and
  2. since LLVM sees more code at once, some super-linear optimizations would take dis-proportionally more time (like when one compiles with LTO now)
• People might rely on pub #[no_mangle] items being exported from RLIBs and link against them directly. This would not be possible anymore, as @nagisa points out.

Non-Advantages

• MIR-only libs would not be platform independent. One could think that that should be the case but because of cfg switches, MIR is not platform independent either.

Mitigation strategies for disadvantages:

1. The problem of caching machine code would be solved in a generalized form by incremental compilation. One has to keep in mind though that incremental compilation will produce less performant code because it prevents many opportunities for inlining.
2. We could provide an additional, more coarse-grained codegen unit partitioning scheme for incremental compilation (e.g. one CGU per crate) for better runtime performance at the cost of longer compile times.
3. The amount of code LLVM sees at once can easily be controlled via -C codegen-units already, which provides a means of reducing super-linear optimizations.

Open Questions

• I think we support "bundling" native libraries into RLIBs. We might still need to keep supporting this, even if we don't store machine code originating from Rust?

Please help collect more data on the viability of MIR-only RLIBs.

cc @rust-lang/core @rust-lang/compiler @rust-lang/tools @rkruppe

[nagisa]

This is also potentially breaking people who are linking to rlibs expecting them to at least expose the extern #[no_mangle] functions like it does currently.

I did that at least once before, though the application where I did it was already very hacky for other reasons and I do not think the project is around anymore.

[japaric]

Another advantage I see is that pure MIR RLIBs effectively let us "recompile"
std with different codegen options without needing std-aware Cargo or Xargo.
This is assuming that the std component that rustup installs will also be
pure MIR.

Basically, cargo rustc --release -- -C target-cpu=native would optimize std
for the host CPU "on the fly". Today, this requires using Xargo to recompile
std (i.e. RUSTFLAGS="-C target-cpu=native xargo build").

Other case where one uses Xargo to recompile the std is producing an
executable that aborts on panic!s without the overhead of landing pads (the
std component that rustup install contains landing pads because it's
compiled with -C panic=unwind). With pure MIR RLIBs, after you set panic = "abort" in your Cargo.toml, cargo build will give you an executable that
doesn't contain landing pads (everything would get compiled with -C panic=abort).

cc @brson ^ pure MIR RLIBs would eliminate the need for std-aware Cargo and
Xargo for some scenarios.

I expect the above will also make using sanitizers (cf #38699)
straightforward. Using a sanitizer requires (re)compiling everything with an
extra LLVM pass and linking to the sanitizer runtime, which is written in C/C++.
With pure MIR RLIBs using a sanitizer would become as simple as cargo rustc -- -Z sanitizer=address; that would compile everything, including std, with the
extra LLVM pass and also link the runtime, which would be provided as an e.g.
librustc_asan.rlib in the std component.

cc @alexcrichton ^ relevant to sanitizer support

I think we support "bundling" native libraries into RLIBs. We might still need to keep supporting this, even if we don't store machine code originating from Rust?

This would be required for the "easy sanitizers" scenario I'm describing above.
Or we could ship the sanitizers as "static libraries", i.e. librustc_asan.a.

Also, note that today one can build statically linked Rust programs using the
MUSL targets without needing to have MUSL installed because libc.a is
embedded inside the std rlib (libstd.rlib) that ships with the std
component.

[nagisa]

@japaric that’s trickly advantage as it prevents us from adding any MIR optimisations that depend on the codegen options set :) We already have one which acts upon the -Zno-landing-pads (ergo --panic=abort)

[est31]

@nagisa @japaric isn't platform independence listed in the issue description as non advantage?

MIR-only libs would not be platform independent. One could think that that should be the case but because of cfg switches, MIR is not platform independent either.

I'd add to the advantages that it would add more parallelism, as the passes up to MIR being finished take less time than passes up to codegen being finished, and in combination with codegen-units, you can now compile the code more in parallel than before. E.g. right now when I bootstrap the compiler, the "whole world" waits for the rustc crate to compile in a single thread. With the change, we wait less, as only its mir has to be available before we can continue. Afterwards when doing the codegen for the binary, we simply can use codegen-units to get the maximum amount of parallelism the hardware gives us.

[japaric]

Could you elaborate on how it would prevent you from adding such optimizations? The way I see it is that the std component will probably continue to be compiled with -C panic=unwind so if you then compile your app with -C panic=abort then LLVM won't be able to optimize as well (or as fast) as if you had recompiled std with -C panic=abort because of the MIR optimizations you mention. However, we would still be better off than today where the std component is shipped filled with landing pads. Or does LLVM always emit landing pads everywhere if the MIR "optimization" you mention is not present? (In that case, it no longer sounds like an optimization but more like a requirement)

If you want the most optimized code possible then, yeah, you would have to use Xargo or std-aware Cargo to opt into MIR optimizations that depend on codegen options. While you are at it you can also throw in --mir-opt-level=3, etc.

[est31]

I agree that MIR optimisations don't really prevent you to have platform agnostic MIR. As both their input and output is MIR, those optimisations could be run in the leaf crates, once the target and other info is known.

However, if earlier stages in the compiler depend on the target, which is the case with cfg, one would either have to refactor the entire compiler to understand cfg's in all later stages, or simulate compilation with all possible combinations of cfg's enabled/disabled (in the end cfg is an on/off question). The first approach will probably hugely bloat code complexity of the compiler, the second approach would bloat runtime complexity exponentially by the number of kinds of used cfg's.

So MIR will probably stay platform dependent for some time.

[japaric]

@est31

isn't platform independence listed in the issue description as non advantage?

I'm not sure what are trying to get at? The -C target-cpu=native optimizations I'm referring to are about LLVM having access to the IR of all functions so it can apply autovectorization, CPU scheduling optimizations, etc. Whereas today -C target-cpu=native is not as good because libstd.rlib already contains machine code that was optimized with -C target-cpu=generic. All these optimizations are "within an architecture", e.g. x86, and after e.g. cfg(target_arch) has taken effect so I'm not sure how "platform agnostic MIR" is related

[nagisa]

@japaric Removing landing pads from MIR is already somewhat a problem since you cannot add them back after fact, so you already lose some of the so-called advantage by being unable to reverse that. Later on we might want to add something more invasive. For a completely hypothetical example consider something resembling autovectorisation which, again, is not exactly reversible and thus -C target-feature=-stuff would become no-op as well. -C debuginfo=2? Stripped to keep binaries smaller because of -C debuginfo=0 before. -C debug-assertions? No-op even without MIR optimisations as debug assertions is essentially a #[cfg].

So, what I’m trying to say is that specifying codegen options on leaf crates only would still not be equivalent (and diverge more over time with extra hypothetical MIR opts) to specifying the codegen option(s) for every crate.

You could (as @est31 did just now) argue for storing unoptimised MIR instead, but that, in addition to inreasing size of intermediate rlibs, serializes MIR opts.

isn't platform independence listed in the issue description as non advantage

Codegen options aren’t exactly related to platform independence in this context.

[japaric]

@michaelwoerister

I'm not sure if this can be listed as an advantage but pure MIR RLIBs would have prevented #38824. The TL;DR is that LLVM raises assertions when you try lower functions that take/return i128 values to PTX code / MPS430 instructions because of bugs in LLVM. With pure MIR RLIBs I expect that if the leaf crate doesn't make use of i128 at all then those functions that use i128 would never be fed into LLVM thus the LLVM assertions wouldn't have been triggered. I suppose that would be some sort of "dead code elimination" pass at the MIR level. So, basically less IR could be fed into LLVM with the right analysis.

[est31]

@japaric

I'm not sure what are trying to get at?

Ah, sorry, I've misread, you only talked about codegen options.

[eddyb]

cc @solson @oli-obk (miri develooers)

[michaelwoerister]

Great points @nagisa, @japaric, and @est31! I've added all of them to the list.

[solson]

I don't think it will affect const evaluation one way or another, but it would help us test Miri outside rustc to be able to easily build dependencies as MIR-only rlibs (with MIR for all items, not just generic/inline/constant ones like in the existing metadata).

Largely, Miri is just like another backend in this context, so it is an instance of this previously mentioned advantage:

There seems be some indication that MIR-only RLIBs would help with making the Rust compiler more backend agnostic (see WASM-related issue #38804).