SIGTRAP during compilation on MacOS

[orlp]

Unfortunately I don't have a minimal reproducible example, just a reproducible example. On MacOS Ventura 13.5 on an Apple M2 processor, the following steps reproduce it:

git clone https://github.com/pola-rs/polars --depth 1
cd polars
make build-opt

Said steps cause rustc to exit with a SIGTRAP signal, unfortunately all the way at the end of compilation.

Meta

The above SIGTRAP is a regression, it occurs with channel = "nightly-2024-09-02" in rust-toolchain.toml, as well as with nightly-2024-08-26, but not with nightly-2024-07-26. I have not bisected it further.

The error does not occur when using make build instead of make build-opt. The difference between these two is the profile selected, make build uses the debug profile whereas built-opt uses the following profile:

[profile.debug-release]
inherits = "release"
debug = true
incremental = true
codegen-units = 16
lto = "thin"

Error output

error: could not compile `py-polars` (lib)

Caused by:
  process didn't exit successfully: `/Users/orlp/.rustup/toolchains/nightly-2024-09-02-aarch64-apple-darwin/bin/rustc --crate-name polars --edition=2021 py-polars/src/lib.rs --error-format=json --json=diagnostic-re
ndered-ansi,artifacts,future-incompat --diagnostic-width=214 --crate-type cdylib --emit=dep-info,link -C opt-level=3 -C lto=thin -C codegen-units=16 -C debuginfo=2 -C split-debuginfo=unpacked -C link-arg=-undefined
 -C link-arg=dynamic_lookup -C 'link-args=-Wl,-install_name,@rpath/polars.abi3.so' --cfg 'feature="all"' --cfg 'feature="build_info"' --cfg 'feature="clipboard"' --cfg 'feature="csv"' --cfg 'feature="default"' --cf
g 'feature="ffi_plugin"' --cfg 'feature="ipc"' --cfg 'feature="nightly"' --cfg 'feature="object"' --cfg 'feature="parquet"' --cfg 'feature="polars_cloud"' --cfg 'feature="sql"' --cfg 'feature="trigonometry"' --chec
k-cfg 'cfg(docsrs)' --check-cfg 'cfg(feature, values("all", "array_any_all", "array_count", "asof_join", "avro", "binary_encoding", "build_info", "clipboard", "cloud", "cross_join", "cse", "csv", "cutqcut", "decomp
ress", "default", "dtype-array", "dtype-i16", "dtype-i8", "dtype-u16", "dtype-u8", "dtypes", "extract_groups", "extract_jsonpath", "ffi_plugin", "find_many", "hist", "io", "ipc", "ipc_streaming", "is_in", "json", "
list_any_all", "list_count", "list_drop_nulls", "list_gather", "list_sample", "list_sets", "merge_sorted", "meta", "new_streaming", "nightly", "object", "operations", "optimizations", "parquet", "pct_change", "peak
s", "performant", "pivot", "polars_cloud", "propagate_nans", "regex", "repeat_by", "rle", "search_sorted", "sign", "sql", "streaming", "timezones", "top_k", "trigonometry"))' -C metadata=ea4eeeedc424a3d4 --out-dir
/Users/orlp/programming/rust/polars/target/debug-release/deps -C incremental=/Users/orlp/programming/rust/polars/target/debug-release/incremental -L dependency=/Users/orlp/programming/rust/polars/target/debug-relea
se/deps --extern jemallocator=/Users/orlp/programming/rust/polars/target/debug-release/deps/libjemallocator-ddd1e6f334ef7c27.rlib --extern libc=/Users/orlp/programming/rust/polars/target/debug-release/deps/liblibc-
42e938498c975329.rlib --extern polars_python=/Users/orlp/programming/rust/polars/target/debug-release/deps/libpolars_python-99e3509ba3962109.rlib --extern pyo3=/Users/orlp/programming/rust/polars/target/debug-relea
se/deps/libpyo3-8bd11066bcba163b.rlib -L native=/Users/orlp/programming/rust/polars/target/debug-release/build/jemalloc-sys-b8e69d721500db60/out/build/lib -L native=/Users/orlp/programming/rust/polars/target/debug-
release/build/lz4-sys-e43c007a0372d905/out -L native=/Users/orlp/programming/rust/polars/target/debug-release/build/ring-ff56c916aa4069c8/out -L native=/Users/orlp/programming/rust/polars/target/debug-release/build
/psm-6c1c5b5e6ef90509/out -L native=/Users/orlp/programming/rust/polars/target/debug-release/build/zstd-sys-a95d2f6c0a1c8f6d/out -L native=/Users/orlp/programming/rust/polars/target/debug-release/build/blake3-44100
89b8e8e51f2/out -L native=/Users/orlp/programming/rust/polars/target/debug-release/build/libz-ng-sys-220b3fe3c720fe48/out/lib -L native=/Users/orlp/programming/rust/polars/target/debug-release/build/libz-ng-sys-220
b3fe3c720fe48/out/lib64 --check-cfg 'cfg(allocator, values("default", "mimalloc"))'` (signal: 5, SIGTRAP: trace/breakpoint trap)
💥 maturin failed
  Caused by: Failed to build a native library through cargo
  Caused by: Cargo build finished with "exit status: 101": `env -u CARGO PYO3_ENVIRONMENT_SIGNATURE="cpython-3.11-64bit" PYO3_PYTHON="/Users/orlp/programming/rust/polars/.venv/bin/python" PYTHON_SYS_EXECUTABLE="/Us
ers/orlp/programming/rust/polars/.venv/bin/python" "cargo" "rustc" "--profile" "debug-release" "--message-format" "json-render-diagnostics" "--manifest-path" "/Users/orlp/programming/rust/polars/py-polars/Cargo.tom
l" "--lib" "--" "-C" "link-arg=-undefined" "-C" "link-arg=dynamic_lookup" "-C" "link-args=-Wl,-install_name,@rpath/polars.abi3.so"`
make: *** [build-opt] Error 1

I've tagged this issue as an 'internal compiler error', even though it does not panic but actually SIGTRAPs. I thus don't have a Rust backtrace, but I did attach lldb to rustc before the sigtrap occurred and got the following backtrace:

Backtrace

Process 62117 stopped
* thread #23, name = 'lto polars_parquet-a4434a04818ab16d.el3w6d8nkur84ww5qq4oo2yuv.r', stop reason = EXC_BREAKPOINT (code=1, subcode=0x10f889e50)
    frame #0: 0x000000010f889e50 librustc_driver-5d5f1ff7b417ce42.dylib`llvm::DataLayout::getTypeSizeInBits(llvm::Type*) const
librustc_driver-5d5f1ff7b417ce42.dylib`llvm::DataLayout::getTypeSizeInBits:
->  0x10f889e50 <+460>: brk    #0x1

librustc_driver-5d5f1ff7b417ce42.dylib`std::__1::vector<llvm::Value*, std::__1::allocator<llvm::Value*>>::insert<llvm::Use*>:
    0x10f889e54 <+0>:   stp    x28, x27, [sp, #-0x60]!
    0x10f889e58 <+4>:   stp    x26, x25, [sp, #0x10]
    0x10f889e5c <+8>:   stp    x24, x23, [sp, #0x20]
Target 0: (rustc) stopped.
(lldb) bt
* thread #23, name = 'lto polars_parquet-a4434a04818ab16d.el3w6d8nkur84ww5qq4oo2yuv.r', stop reason = EXC_BREAKPOINT (code=1, subcode=0x10f889e50)
  * frame #0: 0x000000010f889e50 librustc_driver-5d5f1ff7b417ce42.dylib`llvm::DataLayout::getTypeSizeInBits(llvm::Type*) const
    frame #1: 0x000000010e7ae570 librustc_driver-5d5f1ff7b417ce42.dylib`llvm::slpvectorizer::BoUpSLP::computeMinimumValueSizes()
    frame #2: 0x000000010e7b2a84 librustc_driver-5d5f1ff7b417ce42.dylib`llvm::SLPVectorizerPass::vectorizeStoreChain(llvm::ArrayRef<llvm::Value*>, llvm::slpvectorizer::BoUpSLP&, unsigned int, unsigned int, unsigned int&)
    frame #3: 0x000000010e7b44b8 librustc_driver-5d5f1ff7b417ce42.dylib`llvm::SLPVectorizerPass::vectorizeStores(llvm::ArrayRef<llvm::StoreInst*>, llvm::slpvectorizer::BoUpSLP&, llvm::DenseSet<std::__1::tuple<llvm::Value*, llvm::Value*, llvm::Value*, llvm::Value*, unsigned int>, llvm::DenseMapInfo<std::__1::tuple<llvm::Value*, llvm::Value*, llvm::Value*, llvm::Value*, unsigned int>, void>>&)::$_156::operator()(std::__1::set<std::__1::pair<unsigned int, int>, llvm::SLPVectorizerPass::vectorizeStores(llvm::ArrayRef<llvm::StoreInst*>, llvm::slpvectorizer::BoUpSLP&, llvm::DenseSet<std::__1::tuple<llvm::Value*, llvm::Value*, llvm::Value*, llvm::Value*, unsigned int>, llvm::DenseMapInfo<std::__1::tuple<llvm::Value*, llvm::Value*, llvm::Value*, llvm::Value*, unsigned int>, void>>&)::StoreDistCompare, std::__1::allocator<std::__1::pair<unsigned int, int>>> const&) const
    frame #4: 0x000000010e7b3b30 librustc_driver-5d5f1ff7b417ce42.dylib`llvm::SLPVectorizerPass::vectorizeStores(llvm::ArrayRef<llvm::StoreInst*>, llvm::slpvectorizer::BoUpSLP&, llvm::DenseSet<std::__1::tuple<llvm::Value*, llvm::Value*, llvm::Value*, llvm::Value*, unsigned int>, llvm::DenseMapInfo<std::__1::tuple<llvm::Value*, llvm::Value*, llvm::Value*, llvm::Value*, unsigned int>, void>>&)
    frame #5: 0x000000010e7b05d0 librustc_driver-5d5f1ff7b417ce42.dylib`llvm::SLPVectorizerPass::vectorizeStoreChains(llvm::slpvectorizer::BoUpSLP&)
    frame #6: 0x000000010e7afbac librustc_driver-5d5f1ff7b417ce42.dylib`llvm::SLPVectorizerPass::runImpl(llvm::Function&, llvm::ScalarEvolution*, llvm::TargetTransformInfo*, llvm::TargetLibraryInfo*, llvm::AAResults*, llvm::LoopInfo*, llvm::DominatorTree*, llvm::AssumptionCache*, llvm::DemandedBits*, llvm::OptimizationRemarkEmitter*)
    frame #7: 0x000000010e7af614 librustc_driver-5d5f1ff7b417ce42.dylib`llvm::SLPVectorizerPass::run(llvm::Function&, llvm::AnalysisManager<llvm::Function>&)
    frame #8: 0x000000010f960f48 librustc_driver-5d5f1ff7b417ce42.dylib`llvm::PassManager<llvm::Function, llvm::AnalysisManager<llvm::Function>>::run(llvm::Function&, llvm::AnalysisManager<llvm::Function>&)
    frame #9: 0x000000010f963a2c librustc_driver-5d5f1ff7b417ce42.dylib`llvm::ModuleToFunctionPassAdaptor::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&)
    frame #10: 0x000000010f96033c librustc_driver-5d5f1ff7b417ce42.dylib`llvm::PassManager<llvm::Module, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&)
    frame #11: 0x000000010d5ae318 librustc_driver-5d5f1ff7b417ce42.dylib`LLVMRustOptimize
    frame #12: 0x000000010b1d20b4 librustc_driver-5d5f1ff7b417ce42.dylib`rustc_codegen_llvm::back::write::llvm_optimize
    frame #13: 0x000000010b203d04 librustc_driver-5d5f1ff7b417ce42.dylib`rustc_codegen_llvm::back::lto::run_pass_manager
    frame #14: 0x000000010b204af8 librustc_driver-5d5f1ff7b417ce42.dylib`rustc_codegen_llvm::back::lto::optimize_thin_module
    frame #15: 0x000000010b1c5ca4 librustc_driver-5d5f1ff7b417ce42.dylib`std::sys::backtrace::__rust_begin_short_backtrace::<<rustc_codegen_llvm::LlvmCodegenBackend as rustc_codegen_ssa::traits::backend::ExtraBackendMethods>::spawn_named_thread<rustc_codegen_ssa::back::write::spawn_work<rustc_codegen_llvm::LlvmCodegenBackend>::{closure#0}, ()>::{closure#0}, ()>
    frame #16: 0x000000010b1c9f7c librustc_driver-5d5f1ff7b417ce42.dylib`<<std::thread::Builder>::spawn_unchecked_<<rustc_codegen_llvm::LlvmCodegenBackend as rustc_codegen_ssa::traits::backend::ExtraBackendMethods>::spawn_named_thread<rustc_codegen_ssa::back::write::spawn_work<rustc_codegen_llvm::LlvmCodegenBackend>::{closure#0}, ()>::{closure#0}, ()>::{closure#1} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
    frame #17: 0x000000010d4e5404 librustc_driver-5d5f1ff7b417ce42.dylib`std::sys::pal::unix::thread::Thread::new::thread_start::h4746909631f3e791
    frame #18: 0x000000018b05bfa8 libsystem_pthread.dylib`_pthread_start

[nikic]

@dianqk As this is on M2, can you please take a look?

[dianqk]

Reduced:

target datalayout = "e-m:o-i64:64-i128:128-n32:64-S128-Fn32"
target triple = "arm64-apple-macosx11.0.0"

define void @"_ZN91_$LT$core..slice..iter..Iter$LT$T$GT$$u20$as$u20$core..iter..traits..iterator..Iterator$GT$4fold17h482f4bde4fc3470aE"(ptr %0, i64 %1, i64 %2) personality ptr null {
  br label %4

4:                                                ; preds = %"_ZN4core4iter8adapters3map8map_fold28_$u7b$$u7b$closure$u7d$$u7d$17he00eb30a769dd1bfE.exit", %3
  %5 = phi i64 [ %spec.select.i2.i.i, %"_ZN4core4iter8adapters3map8map_fold28_$u7b$$u7b$closure$u7d$$u7d$17he00eb30a769dd1bfE.exit" ], [ 0, %3 ]
  %spec.select.i.i.i1 = phi i64 [ %spec.select.i.i.i, %"_ZN4core4iter8adapters3map8map_fold28_$u7b$$u7b$closure$u7d$$u7d$17he00eb30a769dd1bfE.exit" ], [ 0, %3 ]
  %6 = invoke ptr null(ptr null, ptr null)
          to label %"_ZN4core4iter8adapters3map8map_fold28_$u7b$$u7b$closure$u7d$$u7d$17he00eb30a769dd1bfE.exit" unwind label %7

7:                                                ; preds = %4
  %8 = landingpad { ptr, i32 }
          cleanup
  %9 = getelementptr i8, ptr %0, i64 48
  %10 = getelementptr i8, ptr %0, i64 40
  store i64 %spec.select.i.i.i1, ptr %10, align 8
  store i64 %5, ptr %9, align 8
  resume { ptr, i32 } zeroinitializer

"_ZN4core4iter8adapters3map8map_fold28_$u7b$$u7b$closure$u7d$$u7d$17he00eb30a769dd1bfE.exit": ; preds = %4
  %trunc.i.i.i = trunc i64 %1 to i1
  %spec.select.i.i.i = select i1 %trunc.i.i.i, i64 40, i64 0
  %trunc.i1.i.i = trunc i64 %5 to i1
  %spec.select.i2.i.i = select i1 %trunc.i1.i.i, i64 0, i64 %2
  br label %4
}

https://llvm.godbolt.org/z/P58q1ozsP

If this reduction is too aggressive, I'll try again tomorrow. :)

[dianqk]

Upstream issue: llvm/llvm-project#107036.