importlib/test_metadata_api fails on Windows buildbots

[jaraco]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Windows10 3.x has failed when building commit 3e0fec7.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/146/builds/4915) and take a look at the build logs.
4. Check if the failure is related to this commit (3e0fec7) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/146/builds/4915

Failed tests:

• test_importlib

Failed subtests:

• test_read_text - test.test_importlib.test_metadata_api.APITests.test_read_text

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

399 tests OK.

10 slowest tests:

• test_math: 6 min 34 sec
• test_asyncio: 4 min 43 sec
• test_tokenize: 3 min 42 sec
• test_multiprocessing_spawn: 3 min 41 sec
• test_lib2to3: 2 min 52 sec
• test_capi: 2 min 30 sec
• test_unparse: 1 min 57 sec
• test_concurrent_futures: 1 min 46 sec
• test_unicodedata: 1 min 42 sec
• test_compileall: 1 min 33 sec

1 test failed:
test_importlib

34 tests skipped:
test_clinic test_curses test_dbm_gnu test_dbm_ndbm test_devpoll
test_epoll test_fcntl test_fork1 test_gdb test_grp test_ioctl
test_kqueue test_multiprocessing_fork
test_multiprocessing_forkserver test_nis test_openpty
test_ossaudiodev test_peg_generator test_perf_profiler test_pipes
test_poll test_posix test_pty test_pwd test_readline test_resource
test_spwd test_syslog test_threadsignals test_wait3 test_wait4
test_xxlimited test_xxtestfuzz test_zipfile64

1 re-run test:
test_importlib

Total duration: 20 min 48 sec

Click to see traceback logs

Traceback (most recent call last):
  File "D:\buildarea\3.x.bolen-windows10\build\Lib\test\test_importlib\test_metadata_api.py", line 86, in test_read_text
    top_level = [
                ^
IndexError: list index out of range

Originally posted by @bedevere-bot in #103584 (comment)

Linked PRs

• gh-103661: Skip failing test on Windows. #103662
• gh-103661: Apply bugfix from importlib_metadata 6.5.1 and restore test. #103681

[jaraco]

Failure message is:

======================================================================
ERROR: test_read_text (test.test_importlib.test_metadata_api.APITests.test_read_text) [egg_with_no_modules-pkg]
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\buildarea\3.x.bolen-windows10\build\Lib\test\test_importlib\test_metadata_api.py", line 86, in test_read_text
    top_level = [
                ^
IndexError: list index out of range
----------------------------------------------------------------------

The tests are failing on Windows buildbots, but I struggle to think of what might be different about the buildbot environments that would affect this test.

I'm also unsure how to replicate the failure. Tests clearly pass on Windows in general. What is it about this test that makes it fail in the buildbot environments only? Since I don't have a way to triage the failure further, I'm going to mark the test as "xfail" for now to stop the buildbot failures.

[jaraco]

Good news is I've been able to replicate the issue on a local build of CPython for Windows, so I ought to be able to inspect for the root cause.

[jaraco]

Interestingly, it seems PackagePath.name is somehow not returning the name only.

> c:\users\jaraco\code\python\cpython\lib\test\test_importlib\test_metadata_api.py(100)test_read_text()
-> top_level = [
(Pdb) files(pkg_name)
[PackagePath('egg_with_no_modules_pkg.egg-info\\PKG-INFO'), PackagePath('egg_with_no_modules_pkg.egg-info\\SOURCES.txt'), PackagePath('egg_with_no_modules_pkg.egg-info\\top_level.txt')]
(Pdb) files(pkg_name)[-1].name
'egg_with_no_modules_pkg.egg-info\\top_level.txt'

[jaraco]

Aah. PackagePath derives from PurePosixPath, so it's not a big surprise that Windows separators aren't honored. What is more of a surprise is that this issue wasn't caught in importlib_metadata or by the Windows CI runs. Why is it only encountered by the buildbots and my local build?

[jaraco]

The issue is here, where Windows-separated paths are generated.

[jaraco]

I also learned that the suppress(Exception) near that line is sufficient to suppress the error. When I put a NameError in there, the tests continued to pass. Probably that trap needs to be less lenient to errors. That may explain why this failure wasn't encountered on CI runs.