INTERNALLERROR when collecting .txt files in v0.23.1

[potiuk]

We seem to be having issued in our CI of Airflow (for example here) after bumping to pytest-asyncio 0.23.1 - it seemed to work fine in 0.23.0)

Example error here: https://github.com/apache/airflow/actions/runs/7079158331/job/19265706175?pr=36017

INTERNALERROR> Traceback (most recent call last):
  INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 271, in wrap_session
  INTERNALERROR>     session.exitstatus = doit(config, session) or 0
  INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 324, in _main
  INTERNALERROR>     config.hook.pytest_collection(session=session)
  INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/_hooks.py", line 493, in __call__
  INTERNALERROR>     return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
  INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/_manager.py", line 115, in _hookexec
  INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/_callers.py", line 152, in _multicall
  INTERNALERROR>     return outcome.get_result()
  INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/_result.py", line 114, in get_result
  INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
  INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/_callers.py", line 77, in _multicall
  INTERNALERROR>     res = hook_impl.function(*args)
  INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 335, in pytest_collection
  INTERNALERROR>     session.perform_collect()
  INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 675, in perform_collect
  INTERNALERROR>     self.items.extend(self.genitems(node))
  INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 845, in genitems
  INTERNALERROR>     yield from self.genitems(subnode)
  INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 842, in genitems
  INTERNALERROR>     rep = collect_one_node(node)
  INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 546, in collect_one_node
  INTERNALERROR>     ihook.pytest_collectstart(collector=collector)
  INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/_hooks.py", line 493, in __call__
  INTERNALERROR>     return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
  INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/_manager.py", line 115, in _hookexec
  INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/_callers.py", line 113, in _multicall
  INTERNALERROR>     raise exception.with_traceback(exception.__traceback__)
  INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/_callers.py", line 77, in _multicall
  INTERNALERROR>     res = hook_impl.function(*args)
  INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pytest_asyncio/plugin.py", line 612, in pytest_collectstart
  INTERNALERROR>     collector.obj.__pytest_asyncio_scoped_event_loop = scoped_event_loop
  INTERNALERROR> AttributeError: 'NoneType' object has no attribute '__pytest_asyncio_scoped_event_loop'

This seems to be happening at collection time, so we are not eaven sure whcih test is causing it. It seems pretty relevant to recent changes in 0.23.0 and 0.23.1, but unforrtunately the message is pretty cryptic and it's hard to say what we should fix.

Any hints there? Or maybe an indication that there is still a bug around the scoped event loop ?

[potiuk]

Seems that the tests are also broken with 0.23.0 . I believe it must have been the incompatibility explained in changelog:

BREAKING: The asyncio_event_loop mark has been removed. Event loops with class, module, package, and session scopes can be requested via the scope keyword argument to the asyncio mark.

But I fail to see how it can affect collection of tests. For now we are limiting pytest-asyncio to < 0.23.0, but any hintss there would useful.

[potiuk]

Actually I manage to bisect it and have a fully reproducible minimal sceario. Seems that the new pytest-asyncio crashes when there is a .txt file added in tested package.

Here is the scenario:

1. Create the venv and installing pytest + pytest-asyncio in it:

[jarek:~/code] % mkdir test-pytest-asyncio
[jarek:~/code] % cd test-pytest-asyncio
[jarek:~/code/test-pytest-asyncio] % pyenv virtualenv 3.11 test-pytest-asyncio
[jarek:~/code/test-pytest-asyncio] 3s % pyenv activate test-pytest-asyncio
pyenv-virtualenv: prompt changing will be removed from future release. configure `export PYENV_VIRTUALENV_DISABLE_PROMPT=1' to simulate the behavior.
[jarek:~/code/test-pytest-asyncio] [test-pytest-asyncio] % pip install pytest pytest-asyncio
Collecting pytest
  Obtaining dependency information for pytest from https://files.pythonhosted.org/packages/f3/8c/f16efd81ca8e293b2cc78f111190a79ee539d0d5d36ccd49975cb3beac60/pytest-7.4.3-py3-none-any.whl.metadata
  Using cached pytest-7.4.3-py3-none-any.whl.metadata (7.9 kB)
Collecting pytest-asyncio
  Obtaining dependency information for pytest-asyncio from Installing collected packages: pluggy, packaging, iniconfig, pytest, pytest-asyncio
..... -> removed for brevity
Successfully installed iniconfig-2.0.0 packaging-23.2 pluggy-1.3.0 pytest-7.4.3 pytest-asyncio-0.23.1

2. Create new package:

[jarek:~/code/test-pytest-asyncio] [test-pytest-asyncio] % mkdir a
[jarek:~/code/test-pytest-asyncio] [test-pytest-asyncio] % touch a/__init__.py

3. Collect tests:

[jarek:~/code/test-pytest-asyncio] [test-pytest-asyncio] 2s % pytest a

platform darwin -- Python 3.11.6, pytest-7.4.3, pluggy-1.3.0
rootdir: /Users/jarek/IdeaProjects/test-pytest-asyncio
plugins: asyncio-0.23.1
asyncio: mode=Mode.STRICT
collected 0 items
= no tests ran in 0.00s =

So far so good.

4. Create a .txt file in the package:

[jarek:~/code/test-pytest-asyncio] [test-pytest-asyncio] 5 % touch a/test.txt

5. Collect tests:

[jarek:~/code/test-pytest-asyncio] [test-pytest-asyncio] % pytest a
platform darwin -- Python 3.11.6, pytest-7.4.3, pluggy-1.3.0
rootdir: /Users/jarek/IdeaProjects/test-pytest-asyncio
plugins: asyncio-0.23.1
asyncio: mode=Mode.STRICT
collected 0 items
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/jarek/.pyenv/versions/3.11.6/envs/test-pytest-asyncio/lib/python3.11/site-packages/_pytest/main.py", line 271, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>                          ^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jarek/.pyenv/versions/3.11.6/envs/test-pytest-asyncio/lib/python3.11/site-packages/_pytest/main.py", line 324, in _main
INTERNALERROR>     config.hook.pytest_collection(session=session)
INTERNALERROR>   File "/Users/jarek/.pyenv/versions/3.11.6/envs/test-pytest-asyncio/lib/python3.11/site-packages/pluggy/_hooks.py", line 493, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jarek/.pyenv/versions/3.11.6/envs/test-pytest-asyncio/lib/python3.11/site-packages/pluggy/_manager.py", line 115, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jarek/.pyenv/versions/3.11.6/envs/test-pytest-asyncio/lib/python3.11/site-packages/pluggy/_callers.py", line 152, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jarek/.pyenv/versions/3.11.6/envs/test-pytest-asyncio/lib/python3.11/site-packages/pluggy/_result.py", line 114, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/Users/jarek/.pyenv/versions/3.11.6/envs/test-pytest-asyncio/lib/python3.11/site-packages/pluggy/_callers.py", line 77, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jarek/.pyenv/versions/3.11.6/envs/test-pytest-asyncio/lib/python3.11/site-packages/_pytest/main.py", line 335, in pytest_collection
INTERNALERROR>     session.perform_collect()
INTERNALERROR>   File "/Users/jarek/.pyenv/versions/3.11.6/envs/test-pytest-asyncio/lib/python3.11/site-packages/_pytest/main.py", line 675, in perform_collect
INTERNALERROR>     self.items.extend(self.genitems(node))
INTERNALERROR>   File "/Users/jarek/.pyenv/versions/3.11.6/envs/test-pytest-asyncio/lib/python3.11/site-packages/_pytest/main.py", line 845, in genitems
INTERNALERROR>     yield from self.genitems(subnode)
INTERNALERROR>   File "/Users/jarek/.pyenv/versions/3.11.6/envs/test-pytest-asyncio/lib/python3.11/site-packages/_pytest/main.py", line 842, in genitems
INTERNALERROR>     rep = collect_one_node(node)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jarek/.pyenv/versions/3.11.6/envs/test-pytest-asyncio/lib/python3.11/site-packages/_pytest/runner.py", line 546, in collect_one_node
INTERNALERROR>     ihook.pytest_collectstart(collector=collector)
INTERNALERROR>   File "/Users/jarek/.pyenv/versions/3.11.6/envs/test-pytest-asyncio/lib/python3.11/site-packages/pluggy/_hooks.py", line 493, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jarek/.pyenv/versions/3.11.6/envs/test-pytest-asyncio/lib/python3.11/site-packages/pluggy/_manager.py", line 115, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jarek/.pyenv/versions/3.11.6/envs/test-pytest-asyncio/lib/python3.11/site-packages/pluggy/_callers.py", line 113, in _multicall
INTERNALERROR>     raise exception.with_traceback(exception.__traceback__)
INTERNALERROR>   File "/Users/jarek/.pyenv/versions/3.11.6/envs/test-pytest-asyncio/lib/python3.11/site-packages/pluggy/_callers.py", line 77, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jarek/.pyenv/versions/3.11.6/envs/test-pytest-asyncio/lib/python3.11/site-packages/pytest_asyncio/plugin.py", line 612, in pytest_collectstart
INTERNALERROR>     collector.obj.__pytest_asyncio_scoped_event_loop = scoped_event_loop
INTERNALERROR>     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> AttributeError: 'NoneType' object has no attribute '__pytest_asyncio_scoped_event_loop'

This is the problem ^^. The same is reproducible with pytest-asyncio==0.23.0

6. Downgrade pytest-asyncio to <0.23.0

[jarek:~/code/test-pytest-asyncio] [test-pytest-asyncio] 1 % pip install "pytest-asyncio<0.23.0"
Collecting pytest-asyncio<0.23.0
  Obtaining dependency information for pytest-asyncio<0.23.0 from https://files.pythonhosted.org/packages/7d/2c/2e5ab8708667972ee31b88bb6fed680ed5ba92dfc2db28e07d0d68d8b3b1/pytest_asyncio-0.21.1-py3-none-any.whl.metadata
  Downloading pytest_asyncio-0.21.1-py3-none-any.whl.metadata (4.0 kB)
Successfully installed pytest-asyncio-0.21.1

7. Collect tetsts

[jarek:~/code/test-pytest-asyncio] [test-pytest-asyncio] % pytest a
platform darwin -- Python 3.11.6, pytest-7.4.3, pluggy-1.3.0
rootdir: /Users/jarek/IdeaProjects/test-pytest-asyncio
plugins: asyncio-0.21.1
asyncio: mode=Mode.STRICT
collected 0 items
= no tests ran in 0.00s =

All good with pytest-asyncio < 0.23.0

[dotlambda]

The same happens when running aiofiles' tests with pytest-asyncio 0.23.1.
It turns out aiofiles has a file named test_file1.txt: https://github.com/Tinche/aiofiles/tree/v23.2.1/tests/resources
Removing the test prefix from the filename fixes the tests.

[seifertm]

Thanks for the report and the investigation!

I can reproduce the issue as well: When pytest encounters a .txt file, it is collected as a DoctestTextFile. Although DoctestTextFile is considered a subtype of Module, its obj property is always None and causes the error you're seeing.

https://github.com/pytest-dev/pytest/blob/714ce2e872f0e1dc3b0363949d084e3e64f88c82/src/_pytest/doctest.py#L422-L423

This is surprising for me as well. Unfortunately, the issue didn't come up in the v0.23 pre-releases, either.

Patch release following shortly.

[seifertm]

@potiuk @dotlambda The issue should be resolved as of v0.23.2

[potiuk]

Thanks for super speedy fix : ) !

[potiuk]

Suggestion: I'd suggest to yank 0.23.0 and 0.23.1 - that's a good way to signal that tha package has some bug that makes it worthy to not be considered by pip and othar tools when resolving dependencies:

More info: https://blog.piwheels.org/new-features-deletion-yanking-and-more/