most of the run-make tests could be silently passing on cases that should fail

[pnkfelix]

Makefiles unfortunately do not error on undefined variable references; they just silently expand into an empty string. And an empty string is also not an invalid command in a recipe.

So, a recipe like this (observe highlighted line): run-make/dylib-chain/Makefile

does not actually work because of this: run-make/tools.mk

because $(call FAIL,main) is going to expand into an empty string; what was wanted was $(call FAILS,main)

Although it looks like most of the tests are using $(call FAIL,xxx), so its probably simpler to just change tools.mk to use the name FAIL instead of FAILS:

% ack FAIL
bootstrap-from-c-with-green/Makefile
11: $(call FAIL,main)

bootstrap-from-c-with-native/Makefile
10: $(call FAIL,main)

c-dynamic-dylib/Makefile
13: $(call FAIL,bar)

c-dynamic-rlib/Makefile
13: $(call FAIL,bar)

c-link-to-rust-dylib/Makefile
9:  $(call FAIL,bar)

c-static-dylib/Makefile
10: $(call FAIL,bar)

dylib-chain/Makefile
12: $(call FAIL,m4)

prefer-dylib/Makefile
8:  $(call FAILS,foo)

tools.mk
13:FAILS = $(TARGET_RPATH_ENV) ( $(RUN_BINFILE) && exit 1 || exit 0 )

(The bigger question is whether attempting to implement this fix is going to actually expose latent bugs...)

[brson]

Yuck. Nice catch.

[pnkfelix]

part of #8058