Error diagnostics for lints are duplicated with `-Z unstable-options`

[jyn514]

I tried this code: https://github.com/rust-lang/rust/blob/573a697a6197abaad5a2d7208dbf1bbc77f4dcf3/src/test/ui/deduplicate-diagnostics.rs

I expected to see this happen: The same error output with and without unstable-options.

Instead, this happened: One of the errors is duplicated three times instead of only twice:

diff --git a/src/test/ui/deduplicate-diagnostics.duplicate.stderr b/src/test/ui/deduplicate-diagnostics.duplicate.stderr
index 3b100b59995..cd4700c7a7c 100644
--- a/src/test/ui/deduplicate-diagnostics.duplicate.stderr
+++ b/src/test/ui/deduplicate-diagnostics.duplicate.stderr
@@ -22,12 +22,6 @@ error[E0452]: malformed lint attribute input
 LL | #[deny("literal")]
    |        ^^^^^^^^^ bad attribute argument
 
-error[E0452]: malformed lint attribute input
-  --> $DIR/deduplicate-diagnostics.rs:8:8
-   |
-LL | #[deny("literal")]
-   |        ^^^^^^^^^ bad attribute argument
-
-error: aborting due to 5 previous errors
+error: aborting due to 4 previous errors
 
 For more information about this error, try `rustc --explain E0452`.

There are many other affected tests, see 3205303 for details.

cc @petrochenkov, do you know what's going wrong here?