Auto merge of rust-lang#139998 - Zalathar:new-executor, r=onur-ozkan

compiletest: Use the new non-libtest executor by default

The new executor was implemented in rust-lang#139660, but required a manual opt-in. This PR activates the new executor by default, but leaves the old libtest-based executor in place (temporarily) to make reverting easier if something unexpectedly goes horribly wrong.

Currently the new executor can be explicitly disabled by passing the `-N` flag to compiletest (e.g. `./x test ui -- -N`), but eventually that flag will be removed, alongside the removal of the libtest dependency. The flag is mostly there to make manual comparative testing easier if something does go wrong.

As before, there *should* be no user-visible difference between the old executor and the new executor.

---

I didn't get much of a response to my [call for testing thread on Zulip](https://rust-lang.zulipchat.com/#narrow/channel/122651-general/topic/Call.20for.20testing.3A.20New.20test.20executor.20for.20compiletest/with/512452105), and the reports I did get (along with my own usage) indicate that there aren't any problems. So I think it's reasonable to move forward with making this the default, in the hopes of being able to remove the libtest dependency relatively soon.

When the libtest dependency is removed, it should be reasonable to build compiletest against pre-built stage0 std by default, even after the stage0 redesign. (Though we should probably have at least one CI job using in-tree stage1 std instead, to guard against the possibility of the `#![feature(internal_output_capture)]` API actually changing.)

Author: bors

src/tools/compiletest/src/common.rs

@@ -415,10 +415,13 @@ pub struct Config {
     /// ABI tests.
     pub minicore_path: Utf8PathBuf,
 
-    /// If true, run tests with the "new" executor that was written to replace
-    /// compiletest's dependency on libtest. Eventually this will become the
-    /// default, and the libtest dependency will be removed.
-    pub new_executor: bool,
+    /// If true, disable the "new" executor, and use the older libtest-based
+    /// executor to run tests instead. This is a temporary fallback, to make
+    /// manual comparative testing easier if bugs are found in the new executor.
+    ///
+    /// FIXME(Zalathar): Eventually remove this flag and remove the libtest
+    /// dependency.
+    pub no_new_executor: bool,
 }
 
 impl Config {

src/tools/compiletest/src/lib.rs

@@ -203,7 +203,7 @@ pub fn parse_config(args: Vec<String>) -> Config {
             "COMMAND",
         )
         .reqopt("", "minicore-path", "path to minicore aux library", "PATH")
-        .optflag("n", "new-executor", "enables the new test executor instead of using libtest")
+        .optflag("N", "no-new-executor", "disables the new test executor, and uses libtest instead")
         .optopt(
             "",
             "debugger",
@@ -450,7 +450,7 @@ pub fn parse_config(args: Vec<String>) -> Config {
 
         minicore_path: opt_path(matches, "minicore-path"),
 
-        new_executor: matches.opt_present("new-executor"),
+        no_new_executor: matches.opt_present("no-new-executor"),
     }
 }
 
@@ -577,9 +577,10 @@ pub fn run_tests(config: Arc<Config>) {
     // Delegate to the executor to filter and run the big list of test structures
     // created during test discovery. When the executor decides to run a test,
     // it will return control to the rest of compiletest by calling `runtest::run`.
-    let res = if config.new_executor {
+    let res = if !config.no_new_executor {
         Ok(executor::run_tests(&config, tests))
     } else {
+        // FIXME(Zalathar): Eventually remove the libtest executor entirely.
         crate::executor::libtest::execute_tests(&config, tests)
     };