os.environ should preserve the case of the OS keys ?

[tzickel]

BPO	28824
Nosy	@pfmoore, @tjguk, @bitdancer, @zware, @eryksun, @zooba, @tzickel

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2016-11-28.18:55:46.277>
labels = ['type-bug', '3.9', '3.10', '3.11', 'library', 'OS-windows']
title = 'os.environ should preserve the case of the OS keys ?'
updated_at = <Date 2022-03-02.11:19:53.537>
user = 'https://github.com/tzickel'

bugs.python.org fields:

activity = <Date 2022-03-02.11:19:53.537>
actor = 'eryksun'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)', 'Windows']
creation = <Date 2016-11-28.18:55:46.277>
creator = 'tzickel'
dependencies = []
files = []
hgrepos = []
issue_num = 28824
keywords = []
message_count = 10.0
messages = ['281906', '281909', '281910', '281912', '281914', '281915', '281918', '387676', '414115', '414141']
nosy_count = 8.0
nosy_names = ['paul.moore', 'tim.golden', 'r.david.murray', 'benrg', 'zach.ware', 'eryksun', 'steve.dower', 'tzickel']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue28824'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

[tzickel]

In Windows, python's os.environ currently handles the case sensitivity different that the OS. While it's true that the OS is case insensitive, it does preserve the case that you first set it as.

For example:
C:\Users\user>set aSD=Blah
C:\Users\user>set asd
aSD=Blah

But in python:
>>> import os
>>> 'aSD' in os.environ.keys()
False

Today as more people pass environment variables to processes, it's better to behave as the OS does. Basically I think that os.environ (both in 2.7 and 3) should preserve the case as well (for when you need to access / iterate over the keys or set a key), but ignore it when you get a key.

cpython/Lib/os.py

Line 733 in b82a5a6

return encode(key).upper()

[bitdancer]

That unfortunately would probably break existing code. It does seem reasonable that case should be ignored on get, though, if the OS does so. So making your 'in' statement work might be acceptable, backward compatibility wise. Probably only in 3.7, though.

Is it the OS that ignores case, or just the shell?

[zooba]

This works fine in Python 3, and also Python 2.7 *unless* you call .keys().

PS D:\> py -2.7
Python 2.7.12 (v2.7.12:d33e0cf91556, Jun 27 2016, 15:24:40) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> 'path' in os.environ, 'path' in os.environ.keys()
(True, False)

PS D:\> py
Python 3.6.0b4 (default, Nov 22 2016, 05:30:12) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> 'path' in os.environ, 'path' in os.environ.keys()
(True, True)

I suspect at this point, we aren't going to change this in Python 2.7 unless someone comes up with an incredibly motivating issue.

[tzickel]

My issue is that somebody wants to pass a few dict like environment variables as some prefix_key=value but he wants to preserve the case of the key for usage in python so the .keys() space needs to be enumerated.

A workaround for this issue can be importing nt and using nt.environ which preserves the cases.

[eryksun]

I've come across a few problems when passing a modified os.environ.copy() to a child process via subprocess.Popen. Ideally it shouldn't be an issue, as long as the OS or C runtime functions are used, but some troublesome programs do their own case-sensitive search for environment variables (definitely a bug). In such cases the simplest workaround is to use nt.environ.copy(), but this doesn't include any changes in the environment since startup.

[tzickel]

Steve, I've checked in Python 3.5.2, and os.environ.keys() still uppercases everything when scanning (for my use case). Has it changed since then ?

[zooba]

Ah, I see what you mean. In this case, we could change how the case-insensitivity is handled here, but it would only be applicable to 3.7. I'm not opposed to changing the default behavior here, but it does kind of bring up the mapping dict discussion again.

If case is important to your application, environment variables are probably the wrong way to go about passing them in anyway. Either use the value of the variable rather than the key, or find a different approach. Given 'nt.environ' is available without case remapping, I think that's the best workaround.

[eryksun]

On Windows, maybe the os.environ mapping could use a case-insensitive subclass of str for its keys, such as the following:

    @total_ordering
    class _CaseInsensitiveString(str):
        def __eq__(self, other):
            if not isinstance(other, str):
                return NotImplemented
            return path.normcase(self) == path.normcase(other)

        def __lt__(self, other):
            if not isinstance(other, str):
                return NotImplemented
            return path.normcase(self) < path.normcase(other)

        def __hash__(self):
            return hash(path.normcase(self))

Change encodekey() to use the above type. For example:

    def encodekey(key):
        return _CaseInsensitiveString(encode(key))

in which encode() is still check_str().