From a5d3ff82d35f5c396e51798b8644fe096e1def05 Mon Sep 17 00:00:00 2001 From: Joshua Nelson Date: Tue, 15 Sep 2020 13:50:17 -0400 Subject: [PATCH 1/3] Improve instructions for adding a new test - Add a section for library tests - Expand section on rustdoc tests --- src/tests/adding.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/tests/adding.md b/src/tests/adding.md index c88d5e81d..c4d84464e 100644 --- a/src/tests/adding.md +++ b/src/tests/adding.md @@ -29,11 +29,12 @@ rough heuristics: - need to run gdb or lldb? use the `debuginfo` test suite - need to inspect LLVM IR or MIR IR? use the `codegen` or `mir-opt` test suites - - need to run rustdoc? Prefer a `rustdoc` test + - need to run rustdoc? Prefer a `rustdoc` or `rustdoc-ui` test. Occasionally you'll need `rustdoc-js` as well. - need to inspect the resulting binary in some way? Then use `run-make` +- Library tests should go in `library/$crate/tests` (where `$crate` is usually `core`, `alloc`, or `std`). - For most other things, [a `ui` (or `ui-fulldeps`) test](#ui) is to be preferred: - - `ui` tests subsume both run-pass, compile-fail, and parse-fail tests + - `ui` tests subsume both `run-pass`, `compile-fail`, and `parse-fail` tests - in the case of warnings or errors, `ui` tests capture the full output, which makes it easier to review but also helps prevent "hidden" regressions in the output From 00b595a6bfc495397fde2d3d06f88673dfeeb85d Mon Sep 17 00:00:00 2001 From: Joshua Nelson Date: Tue, 15 Sep 2020 14:25:43 -0400 Subject: [PATCH 2/3] Address review comments - Say what should be considered a library test - Mention adding error numbers to doc tests - Link to #ui section - Line wrapping --- src/tests/adding.md | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/src/tests/adding.md b/src/tests/adding.md index c4d84464e..819ceda4c 100644 --- a/src/tests/adding.md +++ b/src/tests/adding.md @@ -29,16 +29,25 @@ rough heuristics: - need to run gdb or lldb? use the `debuginfo` test suite - need to inspect LLVM IR or MIR IR? use the `codegen` or `mir-opt` test suites - - need to run rustdoc? Prefer a `rustdoc` or `rustdoc-ui` test. Occasionally you'll need `rustdoc-js` as well. + - need to run rustdoc? Prefer a `rustdoc` or `rustdoc-ui` test. + Occasionally you'll need `rustdoc-js` as well. - need to inspect the resulting binary in some way? Then use `run-make` -- Library tests should go in `library/$crate/tests` (where `$crate` is usually `core`, `alloc`, or `std`). -- For most other things, [a `ui` (or `ui-fulldeps`) test](#ui) is to be - preferred: - - `ui` tests subsume both `run-pass`, `compile-fail`, and `parse-fail` tests +- Library tests should go in `library/${crate}/tests` (where `${crate}` is + usually `core`, `alloc`, or `std`). Library tests include: + - tests that an API behaves properly, including accepting various types or + having some runtime behavior + - tests where any compiler warnings are not relevant to the test + - tests that a use of an API gives a compile error, where the exact error + message is not relevant to the test. These should have an + [error number](`E0XXX`) in the code block to make sure it's the correct error. +- For most other things, [a `ui` (or `ui-fulldeps`) test](#ui) is to be preferred: + - [`ui`](#ui) tests subsume both `run-pass`, `compile-fail`, and `parse-fail` tests - in the case of warnings or errors, `ui` tests capture the full output, which makes it easier to review but also helps prevent "hidden" regressions in the output +[error number]: https://doc.rust-lang.org/rustdoc/unstable-features.html#error-numbers-for-compile-fail-doctests + ## Naming your test We have not traditionally had a lot of structure in the names of From bb72ca7c8d263086780de0b9388b4f971dd9e16e Mon Sep 17 00:00:00 2001 From: Joshua Nelson Date: Wed, 16 Sep 2020 08:43:04 -0400 Subject: [PATCH 3/3] Fix incorrect link It shouldn't have been a link in the first place. Co-authored-by: Yuki Okushi --- src/tests/adding.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/tests/adding.md b/src/tests/adding.md index 819ceda4c..acdc2fd25 100644 --- a/src/tests/adding.md +++ b/src/tests/adding.md @@ -39,7 +39,7 @@ rough heuristics: - tests where any compiler warnings are not relevant to the test - tests that a use of an API gives a compile error, where the exact error message is not relevant to the test. These should have an - [error number](`E0XXX`) in the code block to make sure it's the correct error. + [error number] (`E0XXX`) in the code block to make sure it's the correct error. - For most other things, [a `ui` (or `ui-fulldeps`) test](#ui) is to be preferred: - [`ui`](#ui) tests subsume both `run-pass`, `compile-fail`, and `parse-fail` tests - in the case of warnings or errors, `ui` tests capture the full output,