[8.0] Add support for trusting dev certs on linux (#57108)

* Look up trusted certs consistently on windows (#56701)

* Search for trusted certificates consistently on Windows

1. Don't use thumbprint so we don't get flagged for using SHA-1
2. Make TrustCertificateCore and RemoveCertificateFromTrustedRoots consistent

* Add a note about our usage of Thumbprint on macOS

* Clean up assumptions about root store

* FindBySubjectName expects a string

* Search by serial number to avoid having to parse subject name

* Fix typo

Co-authored-by: Martin Costello <martin@martincostello.com>

* Call DisposeCertificates more consistently

---------

Co-authored-by: Martin Costello <martin@martincostello.com>
(cherry picked from commit ed7ea40)

* Make dev-certs import consistent with kestrel (#57014)

* Make dev-certs import consistent with kestrel

Kestrel checks the subject name and our magic extension - import was only checking the extension.  They can't easily share a method because import has a test hook.

(cherry picked from commit 06155c0)

* Add support for trusting dev certs on linux (#56582)

* Add support for trusting dev certs on linux

There's no consistent way to do this that works for all clients on all Linux distros, but this approach gives us pretty good coverage.  In particular, we aim to support .NET (esp HttpClient), Chromium, and Firefox on Ubuntu- and Fedora-based distros.

Certificate trust is applied per-user, which is simpler and preferable for security reasons, but comes with the notable downside that the process can't be completed within the tool - the user has to update an environment variable, probably in their user profile.  In particular, OpenSSL consumes the `SSL_CERT_DIR` environment variable to determine where it should look for trusted certificates.

We break establishing trust into two categories: OpenSSL, which backs .NET, and NSS databases (henceforth, nssdb), which backs browsers.

To establish trust in OpenSSL, we put the certificate in `~/.dotnet/corefx/cryptography/trusted`, run a simplified version of OpenSSL's `c_rehash` tool on the directory, and ask the user to update `SSL_CERT_DIR`.

To establish trust in nssdb, we search the home directory for Firefox profiles and `~/.pki/nssdb`.  For each one found, we add an entry to the nssdb therein.

Each of these locations (the trusted certificate folder and the list of nssdbs) can be overridden with an environment variable.

This large number of steps introduces a problem that doesn't exist on Windows or macOS - the dev cert can end up trusted by some clients but not by others.  This change introduces a `TrustLevel` concept so that we can produce clearer output when this happens.

The only non-bundled tools required to update certificate trust are `openssl` (the CLI) and `certutil`.  `sudo` is not required, since all changes are within the user's home directory.

* Also trust certificates in the Current User/Root store

A belt-and-suspenders approach for dotnet trust (i.e. in addition to OpenSSL trust) that has the notable advantage of not requiring any environment variables.

* Clarify the mac-specific comments in GetDevelopmentCertificateFromStore

(cherry picked from commit 27ae082)

* Revert 9.0-specific changes

* Restrict permissions to the dev cert directory (#56985)

* Create directories with secure permissions

If we're creating it, make it 700.  If it already exists, warn if it's not 700.

* Don't create a directory specified by the user

(cherry picked from commit 1470e00)

Author: amcasey

src/ProjectTemplates/Shared/DevelopmentCertificate.cs

@@ -35,7 +35,7 @@ private static string EnsureDevelopmentCertificates(string certificatePath, stri
         var manager = CertificateManager.Instance;
         var certificate = manager.CreateAspNetCoreHttpsDevelopmentCertificate(now, now.AddYears(1));
         var certificateThumbprint = certificate.Thumbprint;
-        CertificateManager.ExportCertificate(certificate, path: certificatePath, includePrivateKey: true, certificatePassword, CertificateKeyExportFormat.Pfx);
+        manager.ExportCertificate(certificate, path: certificatePath, includePrivateKey: true, certificatePassword, CertificateKeyExportFormat.Pfx);
 
         return certificateThumbprint;
     }

src/Servers/Kestrel/Core/src/Internal/LoggerExtensions.cs

@@ -40,4 +40,7 @@ internal static partial class LoggerExtensions
 
     [LoggerMessage(8, LogLevel.Warning, "The ASP.NET Core developer certificate is not trusted. For information about trusting the ASP.NET Core developer certificate, see https://aka.ms/aspnet/https-trust-dev-cert.", EventName = "DeveloperCertificateNotTrusted")]
     public static partial void DeveloperCertificateNotTrusted(this ILogger<KestrelServer> logger);
+
+    [LoggerMessage(9, LogLevel.Warning, "The ASP.NET Core developer certificate is only trusted by some clients. For information about trusting the ASP.NET Core developer certificate, see https://aka.ms/aspnet/https-trust-dev-cert", EventName = "DeveloperCertificatePartiallyTrusted")]
+    public static partial void DeveloperCertificatePartiallyTrusted(this ILogger<KestrelServer> logger);
 }

src/Servers/Kestrel/Core/src/KestrelServerOptions.cs

@@ -385,23 +385,34 @@ internal void Serialize(Utf8JsonWriter writer)
                 return null;
             }
 
-            var status = CertificateManager.Instance.CheckCertificateState(cert, interactive: false);
+            var status = CertificateManager.Instance.CheckCertificateState(cert);
             if (!status.Success)
             {
-                // Display a warning indicating to the user that a prompt might appear and provide instructions on what to do in that
-                // case. The underlying implementation of this check is specific to Mac OS and is handled within CheckCertificateState.
-                // Kestrel must NEVER cause a UI prompt on a production system. We only attempt this here because Mac OS is not supported
-                // in production.
+                // Failure is only possible on MacOS and indicates that, if there is a dev cert, it must be from
+                // a dotnet version prior to 7.0 - newer versions store it in such a way that this check succeeds.
+                // (Success does not mean that the dev cert has been trusted).
+                // In practice, success.FailureMessage will always be MacOSCertificateManager.InvalidCertificateState.
+                // Basically, we're just going to encourage the user to generate and trust the dev cert.  We support
+                // these older certificates not by accepting them as-is, but by modernizing them when dev-certs is run.
+                // If we detect an issue here, we can avoid a UI prompt below.
                 Debug.Assert(status.FailureMessage != null, "Status with a failure result must have a message.");
                 logger.DeveloperCertificateFirstRun(status.FailureMessage);
 
                 // Prevent binding to HTTPS if the certificate is not valid (avoid the prompt)
                 return null;
             }
 
-            if (!CertificateManager.Instance.IsTrusted(cert))
+            // On MacOS, this may cause a UI prompt, since it requires accessing the keychain.  Kestrel must NEVER
+            // cause a UI prompt on a production system. We only attempt this here because MacOS is not supported
+            // in production.
+            switch (CertificateManager.Instance.GetTrustLevel(cert))
             {
-                logger.DeveloperCertificateNotTrusted();
+                case CertificateManager.TrustLevel.Partial:
+                    logger.DeveloperCertificatePartiallyTrusted();
+                    break;
+                case CertificateManager.TrustLevel.None:
+                    logger.DeveloperCertificateNotTrusted();
+                    break;
             }
 
             return cert;

src/Servers/Kestrel/Core/src/TlsConfigurationLoader.cs

@@ -176,20 +176,8 @@ public ListenOptions UseHttpsWithSni(
 
     private static bool IsDevelopmentCertificate(X509Certificate2 certificate)
     {
-        if (!string.Equals(certificate.Subject, "CN=localhost", StringComparison.Ordinal))
-        {
-            return false;
-        }
-
-        foreach (var ext in certificate.Extensions)
-        {
-            if (string.Equals(ext.Oid?.Value, CertificateManager.AspNetHttpsOid, StringComparison.Ordinal))
-            {
-                return true;
-            }
-        }
-
-        return false;
+        return string.Equals(certificate.Subject, CertificateManager.LocalhostHttpsDistinguishedName, StringComparison.Ordinal) &&
+            CertificateManager.IsHttpsDevelopmentCertificate(certificate);
     }
 
     private static bool TryGetCertificatePath(string applicationName, [NotNullWhen(true)] out string? path)