Spurious "syntax error in VERSION script" when linking the sanitizers on `dist-x86_64-linux-alt`

[kennytm]

Symptom:

The dist-x86_64-linux-alt job failed when running the build script for rustc_asan in the stage1-std phase.

Sample of the log

[00:30:15] travis_fold:start:stage1-std
travis_time:start:stage1-std
Building stage1 std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
[00:30:15]    Compiling cc v1.0.15
[00:30:15]    Compiling core v0.0.0 (file:///checkout/src/libcore)
[00:30:15]    Compiling build_helper v0.1.0 (file:///checkout/src/build_helper)
[00:30:15]    Compiling unwind v0.0.0 (file:///checkout/src/libunwind)
[00:30:19]    Compiling std v0.0.0 (file:///checkout/src/libstd)
[00:30:23]    Compiling compiler_builtins v0.0.0 (file:///checkout/src/rustc/compiler_builtins_shim)
[00:30:23]    Compiling cmake v0.1.30
[00:30:23]    Compiling alloc_jemalloc v0.0.0 (file:///checkout/src/liballoc_jemalloc)
[00:30:28]    Compiling profiler_builtins v0.0.0 (file:///checkout/src/libprofiler_builtins)
[00:30:28]    Compiling rustc_asan v0.0.0 (file:///checkout/src/librustc_asan)
[00:30:28]    Compiling rustc_msan v0.0.0 (file:///checkout/src/librustc_msan)
[00:30:29]    Compiling rustc_lsan v0.0.0 (file:///checkout/src/librustc_lsan)
[00:30:30]    Compiling rustc_tsan v0.0.0 (file:///checkout/src/librustc_tsan)
[00:30:38] warning: ../libcompiler_builtins/compiler-rt/lib/profile/InstrProfilingMerge.c:86:48: warning: unused parameter 'ProfileSize' [-Wunused-parameter]
[00:30:38] warning:                                       uint64_t ProfileSize) {
[00:30:38] warning:                                                ^
[00:30:38] warning: 1 warning generated.
[00:30:38] warning: ../libcompiler_builtins/compiler-rt/lib/profile/InstrProfilingUtil.c:119:3: warning: implicit declaration of function 'flock' is invalid in C99 [-Wimplicit-function-declaration]
[00:30:38] warning:   flock(fd, LOCK_EX);
[00:30:38] warning:   ^
[00:30:38] warning: ../libcompiler_builtins/compiler-rt/lib/profile/InstrProfilingUtil.c:144:3: warning: implicit declaration of function 'flock' is invalid in C99 [-Wimplicit-function-declaration]
[00:30:38] warning:   flock(fd, LOCK_UN);
[00:30:38] warning:   ^
[00:30:38] warning: 2 warnings generated.
[00:30:38] warning: ../libcompiler_builtins/compiler-rt/lib/profile/InstrProfilingValue.c:108:60: warning: unused parameter 'Data' [-Wunused-parameter]
[00:30:38] warning: static ValueProfNode *allocateOneNode(__llvm_profile_data *Data, uint32_t Index,
[00:30:38] warning:                                                            ^
[00:30:38] warning: ../libcompiler_builtins/compiler-rt/lib/profile/InstrProfilingValue.c:108:75: warning: unused parameter 'Index' [-Wunused-parameter]
[00:30:38] warning: static ValueProfNode *allocateOneNode(__llvm_profile_data *Data, uint32_t Index,
[00:30:38] warning:                                                                           ^
[00:30:38] warning: ../libcompiler_builtins/compiler-rt/lib/profile/InstrProfilingValue.c:109:48: warning: unused parameter 'Value' [-Wunused-parameter]
[00:30:38] warning:                                       uint64_t Value) {
[00:30:38] warning:                                                ^
[00:30:38] warning: 3 warnings generated.
[00:31:40] error: failed to run custom build command for `rustc_asan v0.0.0 (file:///checkout/src/librustc_asan)`
[00:31:40] process didn't exit successfully: `/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/release/build/rustc_asan-e947cddfbde6b9fa/build-script-build` (exit code: 101)
[00:31:40] --- stdout
[00:31:40] cargo:rerun-if-changed=/checkout/src/librustc_asan/../libcompiler_builtins/compiler-rt/CODE_OWNERS.TXT
[00:31:40] cargo:rerun-if-changed=/checkout/src/librustc_asan/../libcompiler_builtins/compiler-rt/CREDITS.TXT
...
<snip>
...
[00:31:42] cargo:rerun-if-changed=/checkout/src/librustc_asan/../libcompiler_builtins/compiler-rt/www/content.css
[00:31:42] cargo:rerun-if-changed=/checkout/src/librustc_asan/../libcompiler_builtins/compiler-rt/www/index.html
[00:31:42] cargo:rustc-link-lib=static=clang_rt.asan-x86_64
[00:31:42] cargo:rustc-link-search=native=/checkout/obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/linux
[00:31:42] running: "cmake" "/checkout/src/librustc_asan/../libcompiler_builtins/compiler-rt" "-DCOMPILER_RT_BUILD_SANITIZERS=ON" "-DCOMPILER_RT_BUILD_BUILTINS=OFF" "-DCOMPILER_RT_BUILD_XRAY=OFF" "-DLLVM_CONFIG_PATH=/checkout/obj/build/x86_64-unknown-linux-gnu/llvm/build/bin/llvm-config" "-DCMAKE_INSTALL_PREFIX=/checkout/obj/build/x86_64-unknown-linux-gnu/native/asan" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC -ffunction-sections -fdata-sections -fPIC --target=x86_64-unknown-linux-gnu --target=x86_64-unknown-linux-gnu" "-DCMAKE_C_COMPILER=/rustroot/bin/clang" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC -ffunction-sections -fdata-sections -fPIC --target=x86_64-unknown-linux-gnu --target=x86_64-unknown-linux-gnu" "-DCMAKE_CXX_COMPILER=/rustroot/bin/clang++" "-DCMAKE_BUILD_TYPE=Release"
[00:31:42] -- The C compiler identification is Clang 6.0.0
[00:31:42] -- The CXX compiler identification is Clang 6.0.0
...
<snip>
...
[00:31:42] -- Performing Test HAS_THREAD_LOCAL
[00:31:42] -- Performing Test HAS_THREAD_LOCAL - Success
[00:31:42] -- Configuring done
[00:31:42] -- Generating done
[00:31:42] -- Build files have been written to: /checkout/obj/build/x86_64-unknown-linux-gnu/native/asan/build
[00:31:42] running: "cmake" "--build" "." "--target" "asan" "--config" "Release" "--"
[00:31:42] Scanning dependencies of target RTAsan_preinit.x86_64
[00:31:42] [  0%] Building CXX object lib/asan/CMakeFiles/RTAsan_preinit.x86_64.dir/asan_preinit.cc.o
[00:31:42] [  0%] Built target RTAsan_preinit.x86_64
[00:31:42] Scanning dependencies of target RTSanitizerCommon.x86_64
[00:31:42] [  0%] Building CXX object lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_allocator.cc.o
[00:31:42] [  0%] Building CXX object lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_common.cc.o
...
<snip>
...
[00:31:43] Scanning dependencies of target RTAsan_dynamic_version_script_dummy.x86_64
[00:31:43] [100%] Building CXX object lib/asan/CMakeFiles/RTAsan_dynamic_version_script_dummy.x86_64.dir/dummy.cc.o
[00:31:43] [100%] Built target RTAsan_dynamic_version_script_dummy.x86_64
[00:31:43]
[00:31:43] --- stderr
[00:31:43] /rustroot/bin/ld:/checkout/obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers:1399: syntax error in VERSION script
[00:31:43] clang-6.0: error: linker command failed with exit code 1 (use -v to see invocation)
[00:31:43] gmake[3]: *** [lib/linux/libclang_rt.asan-i386.so] Error 1
[00:31:43] gmake[2]: *** [lib/asan/CMakeFiles/clang_rt.asan-dynamic-i386.dir/all] Error 2
[00:31:43] gmake[2]: *** Waiting for unfinished jobs....
[00:31:43] gmake[1]: *** [lib/asan/CMakeFiles/asan.dir/rule] Error 2
[00:31:43] gmake: *** [asan] Error 2
[00:31:43] thread 'main' panicked at '
[00:31:43] command did not execute successfully, got: exit code: 2
[00:31:43]
[00:31:43] build script failed, must exit now', /cargo/registry/src/github.com-1ecc6299db9ec823/cmake-0.1.30/src/lib.rs:643:5
[00:31:43] note: Run with `RUST_BACKTRACE=1` for a backtrace.

Previous instances:

• 2018-05-12T04:48:07Z: Don't allocate when creating an empty BTree #50352 (comment)
• 2018-05-19T06:24:09Z: Emit noalias on &mut parameters by default #50744 (comment)

[alexcrichton]

I wonder if this is something like a concurrency related bug in clang? Like maybe a linker script is generated into a temporary location and just happens to collide, causing this to fail sometimes and succeed other times?

Other than that though I have no idea :(

[kennytm]

For reference, the code related to the VERSION script:

1. The invocation: https://github.com/rust-lang/compiler-rt/blob/54d95ca0bd603c322601a61255898e2b8d2a3cae/lib/asan/CMakeLists.txt#L173-L177

2. The cmake macro: https://github.com/rust-lang/compiler-rt/blob/54d95ca0bd603c322601a61255898e2b8d2a3cae/cmake/Modules/SanitizerUtils.cmake#L60-L81

3. The Python script generating the file: https://github.com/rust-lang/compiler-rt/blob/54d95ca0bd603c322601a61255898e2b8d2a3cae/lib/sanitizer_common/scripts/gen_dynamic_list.py

[alexcrichton]

Hm looking again, on the x86_64 builder we shouldn't even be building i386 versions of the library, so we can probably also just stop building those versions and that may fix it

[kennytm]

The content of the *.vers file should look like:

{
global:
  _ZdaPv;
  _ZdaPvRKSt9nothrow_t;
  _ZdaPvSt11align_val_t;
  _ZdaPvSt11align_val_tRKSt9nothrow_t;
  _ZdaPvj;
  _ZdaPvjSt11align_val_t;
  _ZdlPv;
  _ZdlPvRKSt9nothrow_t;
  _ZdlPvSt11align_val_t;
  _ZdlPvSt11align_val_tRKSt9nothrow_t;
  _ZdlPvj;
  _ZdlPvjSt11align_val_t;
  _Znaj;
  _ZnajRKSt9nothrow_t;
  _ZnajSt11align_val_t;
  _ZnajSt11align_val_tRKSt9nothrow_t;
  _Znwj;
  _ZnwjRKSt9nothrow_t;
  _ZnwjSt11align_val_t;
  _ZnwjSt11align_val_tRKSt9nothrow_t;
  __asan_*;
  __cxa_atexit;
  __cxa_throw;
  __fprintf_chk;
  __getdelim;
  __interceptor___cxa_atexit;
  __interceptor___cxa_throw;
  __interceptor___fprintf_chk;
...
<snip>
...
  xdr_u_quad_t;
  xdr_u_short;
  xdr_uint16_t;
  xdr_uint32_t;
  xdr_uint64_t;
  xdr_uint8_t;
  xdrmem_create;
  xdrstdio_create;
local:
  *;
};