Skip to content

Sync rustc_codegen_cranelift #97057

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 30 commits into from
May 15, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
32202f2
Merge commit 'f2cdd4a78d89c009342197cf5844a21f8aa813df' into sync_cg_…
bjorn3 Apr 22, 2022
ea3bbf0
Merge branch 'sync_from_rust'
bjorn3 Apr 22, 2022
7e45747
Rustup to rustc 1.62.0-nightly (082e4ca49 2022-04-26)
bjorn3 Apr 27, 2022
f429b40
Fix running the rustc test suite
bjorn3 Apr 27, 2022
5e0c623
Minor changes to tests.sh
bjorn3 Apr 27, 2022
88b5f57
Use jit_builder.symbol instead of #[no_mangle] for __clif_jit_fn
bjorn3 Feb 14, 2022
944a48d
Remove less of the sysroot build dir on recompilation
bjorn3 Feb 13, 2022
377f44d
Inline ext_config.sh
bjorn3 Feb 13, 2022
617171e
Remove cg_clif_build_sysroot
bjorn3 Feb 13, 2022
2e65a8f
Use -Zcodegen-backend instead of a rustc replacement in cargo-clif
bjorn3 Feb 13, 2022
88d058f
Don't unnecessarily copy rustc_std_workspace_std into the sysroot
bjorn3 Apr 29, 2022
9152ded
Merge pull request #1225 from bjorn3/build_system_rework
bjorn3 Apr 30, 2022
b84daf8
Update actions/checkout to v3
bjorn3 Apr 30, 2022
882c390
Update object for the nightly Cranelift check
bjorn3 Apr 30, 2022
f449c0d
Remove profile overrides for build scripts and their dependencies
bjorn3 Apr 30, 2022
f3fc94f
Fix #[track_caller] with MIR inlining
bjorn3 Apr 30, 2022
f52162f
Fix #[track_caller] location for function chains
bjorn3 Apr 30, 2022
ddde38d
Fix jit mode with cargo-clif wrapper
bjorn3 May 1, 2022
eb881b4
Add missing comma in vscode settings
bjorn3 May 6, 2022
74b9232
Fix assert_assignable for array types
bjorn3 May 6, 2022
acf9398
Rustfmt
bjorn3 May 6, 2022
25f7711
Rustup to rustc 1.62.0-nightly (77652b9ef 2022-05-06)
bjorn3 May 7, 2022
e5f011a
Only enable JIT tests on x86_64
Kmeakin May 9, 2022
533f9d8
Merge pull request #1227 from Kmeakin/master
bjorn3 May 9, 2022
343c532
Use cfg(any()) instead of cfg(disabled)
bjorn3 May 11, 2022
06817d1
Add `unsigned_offset_from` on pointers
scottmcm Apr 9, 2022
bb7b844
Rename `unsigned_offset_from` to `sub_ptr`
scottmcm Apr 9, 2022
e5ff9f1
Sync from rust e1ec3260d79497080ca86540562d410ba67d2a95
bjorn3 May 15, 2022
63734fc
Rustup to rustc 1.62.0-nightly (70b3681bf 2022-05-14)
bjorn3 May 15, 2022
ecd8fa1
Merge commit '63734fcdd718cca089f84c42f3a42c0096cfd431' into sync_cg_…
bjorn3 May 15, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions compiler/rustc_codegen_cranelift/.github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:
timeout-minutes: 10

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3

- name: Install rustfmt
run: |
Expand Down Expand Up @@ -39,7 +39,7 @@ jobs:
TARGET_TRIPLE: aarch64-unknown-linux-gnu

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3

- name: Cache cargo installed crates
uses: actions/cache@v2
Expand Down Expand Up @@ -127,7 +127,7 @@ jobs:
timeout-minutes: 60

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3

#- name: Cache cargo installed crates
# uses: actions/cache@v2
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
timeout-minutes: 60

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3

- name: Cache cargo installed crates
uses: actions/cache@v2
Expand All @@ -34,7 +34,7 @@ jobs:
sed -i 's/cranelift-jit = { version = "\w*.\w*.\w*", optional = true }/cranelift-jit = { git = "https:\/\/github.com\/bytecodealliance\/wasmtime.git", optional = true }/' Cargo.toml
sed -i 's/cranelift-object = "\w*.\w*.\w*"/cranelift-object = { git = "https:\/\/github.com\/bytecodealliance\/wasmtime.git" }/' Cargo.toml

sed -i 's/gimli = { version = "0.25.0", default-features = false, features = \["write"\]}/gimli = { version = "0.26.1", default-features = false, features = ["write"] }/' Cargo.toml
sed -i 's/object = { version = "0.27.0"/object = { version = "0.28.0"/' Cargo.toml

cat Cargo.toml

Expand Down
4 changes: 2 additions & 2 deletions compiler/rustc_codegen_cranelift/.github/workflows/rustc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3

- name: Cache cargo installed crates
uses: actions/cache@v2
Expand Down Expand Up @@ -46,7 +46,7 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3

- name: Cache cargo installed crates
uses: actions/cache@v2
Expand Down
2 changes: 1 addition & 1 deletion compiler/rustc_codegen_cranelift/.vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"rust-analyzer.assist.importEnforceGranularity": true,
"rust-analyzer.assist.importPrefix": "crate",
"rust-analyzer.cargo.runBuildScripts": true,
"rust-analyzer.cargo.features": ["unstable-features"]
"rust-analyzer.cargo.features": ["unstable-features"],
"rust-analyzer.linkedProjects": [
"./Cargo.toml",
//"./build_sysroot/sysroot_src/src/libstd/Cargo.toml",
Expand Down
10 changes: 0 additions & 10 deletions compiler/rustc_codegen_cranelift/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -41,15 +41,5 @@ unstable-features = ["jit", "inline_asm"]
jit = ["cranelift-jit", "libloading"]
inline_asm = []

# Disable optimizations and debuginfo of build scripts and some of the heavy build deps, as the
# execution time of build scripts is so fast that optimizing them slows down the total build time.
[profile.release.build-override]
opt-level = 0
debug = false

[profile.release.package.cranelift-codegen-meta]
opt-level = 0
debug = false

[package.metadata.rust-analyzer]
rustc_private = true
12 changes: 6 additions & 6 deletions compiler/rustc_codegen_cranelift/build_sysroot/Cargo.lock
Original file line number Diff line number Diff line change
Expand Up @@ -112,9 +112,9 @@ dependencies = [

[[package]]
name = "hashbrown"
version = "0.12.0"
version = "0.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8c21d40587b92fa6a6c6e3c1bdbf87d75511db5672f9c93175574b3a00df1758"
checksum = "db0d4cf898abf0081f964436dc980e96670a0f36863e4b83aaacdb65c9d7ccc3"
dependencies = [
"compiler_builtins",
"rustc-std-workspace-alloc",
Expand All @@ -134,18 +134,18 @@ dependencies = [

[[package]]
name = "libc"
version = "0.2.124"
version = "0.2.125"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "21a41fed9d98f27ab1c6d161da622a4fa35e8a54a8adc24bbf3ddd0ef70b0e50"
checksum = "5916d2ae698f6de9bfb891ad7a8d65c09d232dc58cc4ac433c7da3b2fd84bc2b"
dependencies = [
"rustc-std-workspace-core",
]

[[package]]
name = "memchr"
version = "2.4.1"
version = "2.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "308cc39be01b73d0d18f82a0e7b2a3df85245f84af96fdddc5d202d27e47b86a"
checksum = "2dffe52ecf27772e601905b7522cb4ef790d2cc203488bbd0e2fe85fcb74566d"
dependencies = [
"compiler_builtins",
"rustc-std-workspace-core",
Expand Down
12 changes: 0 additions & 12 deletions compiler/rustc_codegen_cranelift/build_system/build_backend.rs
Original file line number Diff line number Diff line change
Expand Up @@ -34,18 +34,6 @@ pub(crate) fn build_backend(
_ => unreachable!(),
}

// Set the rpath to make the cg_clif executable find librustc_codegen_cranelift without changing
// LD_LIBRARY_PATH
if cfg!(unix) {
if cfg!(target_os = "macos") {
rustflags += " -Csplit-debuginfo=unpacked \
-Clink-arg=-Wl,-rpath,@loader_path/../lib \
-Zosx-rpath-install-name";
} else {
rustflags += " -Clink-arg=-Wl,-rpath=$ORIGIN/../lib ";
}
}

cmd.env("RUSTFLAGS", rustflags);

eprintln!("[BUILD] rustc_codegen_cranelift");
Expand Down
86 changes: 44 additions & 42 deletions compiler/rustc_codegen_cranelift/build_system/build_sysroot.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
use std::env;
use std::fs;
use std::path::{Path, PathBuf};
use std::process::{self, Command};
Expand All @@ -22,35 +21,28 @@ pub(crate) fn build_sysroot(
fs::create_dir_all(target_dir.join("lib")).unwrap();

// Copy the backend
for file in ["cg_clif", "cg_clif_build_sysroot"] {
try_hard_link(
cg_clif_build_dir.join(get_file_name(file, "bin")),
target_dir.join("bin").join(get_file_name(file, "bin")),
);
}

let cg_clif_dylib = get_file_name("rustc_codegen_cranelift", "dylib");
try_hard_link(
cg_clif_build_dir.join(&cg_clif_dylib),
target_dir
.join(if cfg!(windows) {
// Windows doesn't have rpath support, so the cg_clif dylib needs to be next to the
// binaries.
"bin"
} else {
"lib"
})
.join(cg_clif_dylib),
);

// Build and copy cargo wrapper
let mut build_cargo_wrapper_cmd = Command::new("rustc");
build_cargo_wrapper_cmd
.arg("scripts/cargo-clif.rs")
.arg("-o")
.arg(target_dir.join("cargo-clif"))
.arg("-g");
spawn_and_wait(build_cargo_wrapper_cmd);
let cg_clif_dylib_path = target_dir
.join(if cfg!(windows) {
// Windows doesn't have rpath support, so the cg_clif dylib needs to be next to the
// binaries.
"bin"
} else {
"lib"
})
.join(&cg_clif_dylib);
try_hard_link(cg_clif_build_dir.join(cg_clif_dylib), &cg_clif_dylib_path);

// Build and copy rustc and cargo wrappers
for wrapper in ["rustc-clif", "cargo-clif"] {
let mut build_cargo_wrapper_cmd = Command::new("rustc");
build_cargo_wrapper_cmd
.arg(PathBuf::from("scripts").join(format!("{wrapper}.rs")))
.arg("-o")
.arg(target_dir.join(wrapper))
.arg("-g");
spawn_and_wait(build_cargo_wrapper_cmd);
}

let default_sysroot = super::rustc_info::get_default_sysroot();

Expand Down Expand Up @@ -117,7 +109,13 @@ pub(crate) fn build_sysroot(
}
}
SysrootKind::Clif => {
build_clif_sysroot_for_triple(channel, target_dir, host_triple, None);
build_clif_sysroot_for_triple(
channel,
target_dir,
host_triple,
&cg_clif_dylib_path,
None,
);

if host_triple != target_triple {
// When cross-compiling it is often necessary to manually pick the right linker
Expand All @@ -126,14 +124,21 @@ pub(crate) fn build_sysroot(
} else {
None
};
build_clif_sysroot_for_triple(channel, target_dir, target_triple, linker);
build_clif_sysroot_for_triple(
channel,
target_dir,
target_triple,
&cg_clif_dylib_path,
linker,
);
}

// Copy std for the host to the lib dir. This is necessary for the jit mode to find
// libstd.
for file in fs::read_dir(host_rustlib_lib).unwrap() {
let file = file.unwrap().path();
if file.file_name().unwrap().to_str().unwrap().contains("std-") {
let filename = file.file_name().unwrap().to_str().unwrap();
if filename.contains("std-") && !filename.contains(".rlib") {
try_hard_link(&file, target_dir.join("lib").join(file.file_name().unwrap()));
}
}
Expand All @@ -145,6 +150,7 @@ fn build_clif_sysroot_for_triple(
channel: &str,
target_dir: &Path,
triple: &str,
cg_clif_dylib_path: &Path,
linker: Option<&str>,
) {
match fs::read_to_string(Path::new("build_sysroot").join("rustc_version")) {
Expand All @@ -168,18 +174,18 @@ fn build_clif_sysroot_for_triple(
let build_dir = Path::new("build_sysroot").join("target").join(triple).join(channel);

if !super::config::get_bool("keep_sysroot") {
// Cleanup the target dir with the exception of build scripts and the incremental cache
for dir in ["build", "deps", "examples", "native"] {
if build_dir.join(dir).exists() {
fs::remove_dir_all(build_dir.join(dir)).unwrap();
}
// Cleanup the deps dir, but keep build scripts and the incremental cache for faster
// recompilation as they are not affected by changes in cg_clif.
if build_dir.join("deps").exists() {
fs::remove_dir_all(build_dir.join("deps")).unwrap();
}
}

// Build sysroot
let mut build_cmd = Command::new("cargo");
build_cmd.arg("build").arg("--target").arg(triple).current_dir("build_sysroot");
let mut rustflags = "--clif -Zforce-unstable-if-unmarked".to_string();
let mut rustflags = "-Zforce-unstable-if-unmarked -Cpanic=abort".to_string();
rustflags.push_str(&format!(" -Zcodegen-backend={}", cg_clif_dylib_path.to_str().unwrap()));
if channel == "release" {
build_cmd.arg("--release");
rustflags.push_str(" -Zmir-opt-level=3");
Expand All @@ -189,10 +195,6 @@ fn build_clif_sysroot_for_triple(
write!(rustflags, " -Clinker={}", linker).unwrap();
}
build_cmd.env("RUSTFLAGS", rustflags);
build_cmd.env(
"RUSTC",
env::current_dir().unwrap().join(target_dir).join("bin").join("cg_clif_build_sysroot"),
);
build_cmd.env("__CARGO_DEFAULT_LIB_METADATA", "cg_clif");
spawn_and_wait(build_cmd);

Expand Down
1 change: 1 addition & 0 deletions compiler/rustc_codegen_cranelift/build_system/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ pub fn main() {
arg => arg_error!("Unexpected argument {}", arg),
}
}
target_dir = std::env::current_dir().unwrap().join(target_dir);

let host_triple = if let Ok(host_triple) = std::env::var("HOST_TRIPLE") {
host_triple
Expand Down
6 changes: 3 additions & 3 deletions compiler/rustc_codegen_cranelift/docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ This will build your project with rustc_codegen_cranelift instead of the usual L
> You should prefer using the Cargo method.

```bash
$ $cg_clif_dir/build/bin/cg_clif my_crate.rs
$ $cg_clif_dir/build/rustc-clif my_crate.rs
```

## Jit mode
Expand All @@ -38,7 +38,7 @@ $ $cg_clif_dir/build/cargo-clif jit
or

```bash
$ $cg_clif_dir/build/bin/cg_clif -Zunstable-features -Cllvm-args=mode=jit -Cprefer-dynamic my_crate.rs
$ $cg_clif_dir/build/rustc-clif -Zunstable-features -Cllvm-args=mode=jit -Cprefer-dynamic my_crate.rs
```

There is also an experimental lazy jit mode. In this mode functions are only compiled once they are
Expand All @@ -54,7 +54,7 @@ These are a few functions that allow you to easily run rust code from the shell

```bash
function jit_naked() {
echo "$@" | $cg_clif_dir/build/bin/cg_clif - -Zunstable-features -Cllvm-args=mode=jit -Cprefer-dynamic
echo "$@" | $cg_clif_dir/build/rustc-clif - -Zunstable-features -Cllvm-args=mode=jit -Cprefer-dynamic
}

function jit() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ index 092b7cf..158cf71 100644
-#[cfg(target_has_atomic_load_store = "128")]
-#[unstable(feature = "integer_atomics", issue = "32976")]
-impl RefUnwindSafe for crate::sync::atomic::AtomicI128 {}

#[cfg(target_has_atomic_load_store = "ptr")]
#[stable(feature = "unwind_safe_atomic_refs", since = "1.14.0")]
@@ -235,9 +232,6 @@ impl RefUnwindSafe for crate::sync::atomic::AtomicU32 {}
Expand All @@ -31,14 +31,14 @@ index 092b7cf..158cf71 100644
-#[cfg(target_has_atomic_load_store = "128")]
-#[unstable(feature = "integer_atomics", issue = "32976")]
-impl RefUnwindSafe for crate::sync::atomic::AtomicU128 {}

#[cfg(target_has_atomic_load_store = "8")]
#[stable(feature = "unwind_safe_atomic_refs", since = "1.14.0")]
diff --git a/library/core/src/sync/atomic.rs b/library/core/src/sync/atomic.rs
index d9de37e..8293fce 100644
--- a/library/core/src/sync/atomic.rs
+++ b/library/core/src/sync/atomic.rs
@@ -2234,44 +2234,6 @@ atomic_int! {
@@ -2234,46 +2234,6 @@ atomic_int! {
"AtomicU64::new(0)",
u64 AtomicU64 ATOMIC_U64_INIT
}
Expand All @@ -54,6 +54,7 @@ index d9de37e..8293fce 100644
- unstable(feature = "integer_atomics", issue = "32976"),
- rustc_const_stable(feature = "const_integer_atomics", since = "1.34.0"),
- unstable(feature = "integer_atomics", issue = "32976"),
- cfg_attr(not(test), rustc_diagnostic_item = "AtomicI128"),
- "i128",
- "#![feature(integer_atomics)]\n\n",
- atomic_min, atomic_max,
Expand All @@ -73,6 +74,7 @@ index d9de37e..8293fce 100644
- unstable(feature = "integer_atomics", issue = "32976"),
- rustc_const_stable(feature = "const_integer_atomics", since = "1.34.0"),
- unstable(feature = "integer_atomics", issue = "32976"),
- cfg_attr(not(test), rustc_diagnostic_item = "AtomicU128"),
- "u128",
- "#![feature(integer_atomics)]\n\n",
- atomic_umin, atomic_umax,
Expand All @@ -98,6 +100,6 @@ index b735957..ea728b6 100644
#[cfg(target_has_atomic = "ptr")]
assert_eq!(align_of::<AtomicUsize>(), size_of::<AtomicUsize>());
#[cfg(target_has_atomic = "ptr")]
--
--
2.26.2.7.g19db9cfb68

2 changes: 1 addition & 1 deletion compiler/rustc_codegen_cranelift/rust-toolchain
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[toolchain]
channel = "nightly-2022-04-21"
channel = "nightly-2022-05-15"
components = ["rust-src", "rustc-dev", "llvm-tools-preview"]
Loading