`subprocess.run` docs should recommend copying `os.environ` on Windows

[ncoghlan]

As described in https://stackoverflow.com/questions/78652758/, when passing custom environment variables to subprocess.run or subprocess.Popen on Windows, you will usually need to include the existing keys from os.environ in order to reliably run arbitrary processes (exactly which keys are required to be present depends on what the subprocess is doing).

The subprocess.Popen docs include a "Note" about this problem, but the subprocess.run docs omit it. Even the subprocess.Popen note is a bit vague, and only specifically mentions SystemRoot:

Note: If specified, env must provide any variables required for the program to execute. On Windows, in order to run a side-by-side assembly the specified env must include a valid SystemRoot.

The common documentation for the env parameter reads:

If env is not None, it must be a mapping that defines the environment variables for the new process; these are used instead of the default behavior of inheriting the current process’ environment. It is passed directly to Popen. This mapping can be str to str on any platform or bytes to bytes on POSIX platforms much like os.environ or os.environb.

Rather than re-using the existing subprocess.Popen note, my suggestion would be to add the following sentence to the above paragraph:

On Windows, many applications require specific environment variables (such as SystemRoot) to be set as expected in order to work correctly. Accordingly, it is recommended to start from os.environ.copy() when customing the subprocess environment on Windows, rather than starting from an empty dictionary (otherwise the executed subprocesses may fail with various Windows OS errors).

[eryksun]

I know that "SystemRoot" and "SystemDrive" are required, even just within the Windows API, because some DLL and other resource paths in the registry are specified as REG_EXPAND_SZ strings that use them. Such paths are usually expanded using WinAPI ExpandEnvironmentStringsW(). For some reason, this API function doesn't get a fallback value from GetWindowsDirectoryW(), or even by reading the "SystemRoot" value in "HKLM\Software\Microsoft\Windows NT\Currentversion". For example:

>>> from win32api import ExpandEnvironmentStrings, GetWindowsDirectory
>>> ExpandEnvironmentStrings('%SystemRoot%')
'C:\\Windows'
>>> del os.environ['SystemRoot']
>>> ExpandEnvironmentStrings('%SystemRoot%')
'%SystemRoot%'
>>> GetWindowsDirectory() # it could use this, but doesn't
'C:\\Windows'

"SystemRoot" and "SystemDrive" are also required by CreateEnvironmentBlock(), since for whatever reason it always copies them from the current environment instead of getting their values from the API or registry. Since "SystemRoot" and "SystemDrive" are crucial for expanding the REG_EXPAND_SZ values of many other environment variables, the new environment is totally broken if they aren't set before calling CreateEnvironmentBlock().

[ncoghlan]

As far as I'm aware, there's also no guarantee that Microsoft won't add other required environment variables in the future, so "copy the existing environment" seems like the only future-proof approach.

[eryksun]

For sure, I'm just filling in the background reason why "SystemRoot" being missing is usually a problem, that ExpandEnvironmentStringsW() doesn't fall back on a default value from the API or registry. Also, in places where we explicitly recommend keeping "SystemRoot", we should also recommend "SystemDrive" since they're both needed in general. If we change all cases to recommend copying os.environ, then that's not an issue.

[jaraco]

"copy the existing environment" seems like the only future-proof approach.

What if someone wants to create a "clean" or "isolated" environment? os.environ is unlikely to be that. Perhaps better would be for Python (or a library) to provide and maintain a mask of the environment with known required keys. Should Microsoft add required ones, the mask could be updated.