CI: Builds failing in ``pandas.show_versions()``

[johnzangwill]

Failing on all platofrms. E.g.
https://github.com/pandas-dev/pandas/runs/4581277621

Run python -c "import pandas; pandas.show_versions();"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/runner/work/pandas/pandas/pandas/util/_print_versions.py", line 109, in show_versions
    deps = _get_dependency_info()
  File "/home/runner/work/pandas/pandas/pandas/util/_print_versions.py", line 88, in _get_dependency_info
    mod = import_optional_dependency(modname, errors="ignore")
  File "/home/runner/work/pandas/pandas/pandas/compat/_optional.py", line 115, in import_optional_dependency
    module = importlib.import_module(name)
  File "/opt/hostedtoolcache/Python/3.10.1/x64/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1002, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 945, in _find_spec
  File "/opt/hostedtoolcache/Python/3.10.1/x64/lib/python3.10/site-packages/_distutils_hack/__init__.py", line 83, in find_spec
    return method()
  File "/opt/hostedtoolcache/Python/3.10.1/x64/lib/python3.10/site-packages/_distutils_hack/__init__.py", line 104, in spec_for_pip
    if self.pip_imported_during_build():
  File "/opt/hostedtoolcache/Python/3.10.1/x64/lib/python3.10/site-packages/_distutils_hack/__init__.py", line 115, in pip_imported_during_build
    return any(
  File "/opt/hostedtoolcache/Python/3.10.1/x64/lib/python3.10/site-packages/_distutils_hack/__init__.py", line 116, in <genexpr>
    frame.f_globals['__file__'].endswith('setup.py')
KeyError: '__file__'
Error: Process completed with exit code 1.

[jreback]

yeah maybe something on a pip release?

cc @pandas-dev/pandas-core

[MarcoGorelli]

I was looking into this earlier and couldn't reproduce it, even in a clean conda environment with the latest setuptools and wheel, couldn't even find other issues about it

[phofl]

pypa/setuptools#2941

[jreback]

looks like 60.0.3 was released to fix this

[phofl]

60.0.3 did not fix all problems for us unfortunately. When running this locally I get

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/testenv2/lib/python3.10/site-packages/pandas/util/_print_versions.py", line 109, in show_versions
    deps = _get_dependency_info()
  File "/tmp/testenv2/lib/python3.10/site-packages/pandas/util/_print_versions.py", line 88, in _get_dependency_info
    mod = import_optional_dependency(modname, errors="ignore")
  File "/tmp/testenv2/lib/python3.10/site-packages/pandas/compat/_optional.py", line 115, in import_optional_dependency
    module = importlib.import_module(name)
  File "/tmp/testenv2/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/tmp/testenv2/lib/python3.10/site-packages/setuptools/__init__.py", line 8, in <module>
    import _distutils_hack.override  # noqa: F401
  File "/tmp/testenv2/lib/python3.10/site-packages/_distutils_hack/override.py", line 1, in <module>
    __import__('_distutils_hack').do_override()
  File "/tmp/testenv2/lib/python3.10/site-packages/_distutils_hack/__init__.py", line 76, in do_override
    ensure_local_distutils()
  File "/tmp/testenv2/lib/python3.10/site-packages/_distutils_hack/__init__.py", line 63, in ensure_local_distutils
    assert '_distutils' in core.__file__, core.__file__
AssertionError: /tmp/testenv2/lib/python3.10/distutils/core.py

It looks like it tries to distutils from stdlib, but if running

import importlib
importlib.import_module("setuptools")

it is pulling the distutils from

'/tmp/testenv2/lib/python3.10/site-packages/setuptools/_distutils'

Not sure what is different when doing this via pandas.show_versions()

[lithomas1]

This happens because show_version() imports all the dependencies installed including pip and setuptools. Apparently importing pip before setuptools causes this to happen. Swapping these too lines seems to fix it for me.

pandas/pandas/util/_print_versions.py

Lines 63 to 64 in e18921e

	"pip",
	"setuptools",

Perhaps its better to wait for an upstream fix, though?

[mgorny]

I couldn't find an upstream bug report about this and I've reproduced it with git HEAD, so I've filed pypa/setuptools#3044.

[mgorny]

Perhaps its better to wait for an upstream fix, though?

Upstream has closed the bug as wontfix. Basically, pip has weird internal expectations that they can't change right now and importing pip is discouraged anyway.

[MarcoGorelli]

Probably fine to just remove pip from the list of packages to print the version of then?

[max-uzunov]

To workaround this issue one can set an OS environment variable: SETUPTOOLS_USE_DISTUTILS=stdlib - this should work for setuptools v60+.

pypa/setuptools#2965