FileNotFoundError while writing results against a test suite that modifies current working directory

[simonw]

Describe the bug

When running 5.0a5 against my datasette project I get the following error at the end of the test run:

tests/test_publish_now.py .......                                                                                        [ 81%]
tests/test_utils.py .................................................................                                    [100%]
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File ".../site-packages/_pytest/main.py", line 213, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File ".../site-packages/_pytest/main.py", line 257, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File ".../site-packages/pluggy/hooks.py", line 289, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File ".../site-packages/pluggy/manager.py", line 87, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File ".../site-packages/pluggy/manager.py", line 81, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File ".../site-packages/pluggy/callers.py", line 203, in _multicall
INTERNALERROR>     gen.send(outcome)
INTERNALERROR>   File ".../site-packages/pytest_cov/plugin.py", line 229, in pytest_runtestloop
INTERNALERROR>     self.cov_controller.finish()
INTERNALERROR>   File ".../site-packages/pytest_cov/engine.py", line 171, in finish
INTERNALERROR>     self.cov.stop()
INTERNALERROR>   File ".../site-packages/coverage/control.py", line 553, in save
INTERNALERROR>     data = self.get_data()
INTERNALERROR>   File ".../site-packages/coverage/control.py", line 607, in get_data
INTERNALERROR>     if self._collector and self._collector.flush_data():
INTERNALERROR>   File ".../site-packages/coverage/collector.py", line 425, in flush_data
INTERNALERROR>     self.covdata.add_lines(abs_file_dict(self.data))
INTERNALERROR>   File ".../site-packages/coverage/sqldata.py", line 245, in add_lines
INTERNALERROR>     self._choose_lines_or_arcs(lines=True)
INTERNALERROR>   File ".../site-packages/coverage/sqldata.py", line 288, in _choose_lines_or_arcs
INTERNALERROR>     with self._connect() as con:
INTERNALERROR>   File ".../site-packages/coverage/sqldata.py", line 161, in _connect
INTERNALERROR>     self._create_db()
INTERNALERROR>   File ".../site-packages/coverage/sqldata.py", line 118, in _create_db
INTERNALERROR>     with self._dbs[get_thread_id()] as db:
INTERNALERROR>   File ".../site-packages/coverage/sqldata.py", line 616, in __enter__
INTERNALERROR>     self.connect()
INTERNALERROR>   File ".../site-packages/coverage/sqldata.py", line 601, in connect
INTERNALERROR>     filename = os.path.relpath(self.filename)
INTERNALERROR>   File ".../lib/python3.7/posixpath.py", line 475, in relpath
INTERNALERROR>     start_list = [x for x in abspath(start).split(sep) if x]
INTERNALERROR>   File ".../lib/python3.7/posixpath.py", line 383, in abspath
INTERNALERROR>     cwd = os.getcwd()
INTERNALERROR> FileNotFoundError: [Errno 2] No such file or directory

=========== 355 passed in 47.88 seconds =========

I'm pretty sure this is because some of my tests in that test suite mess around with the current working directory. It would be great if coverage.py was robust against changes made to the working directory.

To Reproduce

These steps to reproduce work on my OS X laptop running Python 3.7:

cd /tmp
mkdir coverage-error
cd coverage-error/
pipenv shell
git clone https://github.com/simonw/datasette
cd datasette/
pip install -e .[test]
pip install coverage==5.0a5 pytest-cov
pytest --cov=datasette

[simonw]

I'm not entirely sure, but I'm suspicious that the test behaviour which triggers this is my use of the Click .isolated_filesystem() testing mechanism: http://click.palletsprojects.com/en/5.x/testing/#file-system-isolation

I use that here (and in a bunch of other places): https://github.com/simonw/datasette/blob/a9453c4dda70bbf5122835e68f63db6ecbe1a6fc/tests/test_publish_now.py#L118-L119

[simonw]

It looks like this bug is NOT unique to the 5.0 series.

I ran pip install coverage==4.5.3 in that same testing environment and then ran pytest --cov=datasette again and got this, similar error:

INTERNALERROR>   File ".../site-packages/pytest_cov/plugin.py", line 229, in pytest_runtestloop
INTERNALERROR>     self.cov_controller.finish()
INTERNALERROR>   File ".../site-packages/pytest_cov/engine.py", line 171, in finish
INTERNALERROR>     self.cov.stop()
INTERNALERROR>   File ".../site-packages/coverage/control.py", line 675, in load
INTERNALERROR>     self._init()
INTERNALERROR>   File ".../site-packages/coverage/control.py", line 223, in _init
INTERNALERROR>     set_relative_directory()
INTERNALERROR>   File ".../site-packages/coverage/files.py", line 28, in set_relative_directory
INTERNALERROR>     RELATIVE_DIR = os.path.normcase(abs_file(os.curdir) + os.sep)
INTERNALERROR>   File ".../site-packages/coverage/files.py", line 163, in abs_file
INTERNALERROR>     path = os.path.realpath(path)
INTERNALERROR>   File ".../lib/python3.7/posixpath.py", line 396, in realpath
INTERNALERROR>     return abspath(path)
INTERNALERROR>   File ".../lib/python3.7/posixpath.py", line 383, in abspath
INTERNALERROR>     cwd = os.getcwd()
INTERNALERROR> FileNotFoundError: [Errno 2] No such file or directory

[nedbat]

Something is in the air... this was fixed on master last week: #806 . I have no idea why it hasn't been reported before.

[simonw]

Unfortunately I don't think it's fixed in master - I tried running against master by using the following:

pip install https://github.com/nedbat/coveragepy/archive/master.zip

Then running pytest --cov=datasette again... but I still got the same error:

datasette/tests/test_utils.py .................................................................                                                                             [100%]
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File ".../site-packages/_pytest/main.py", line 213, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File ".../site-packages/_pytest/main.py", line 257, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File ".../site-packages/pluggy/hooks.py", line 289, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File ".../site-packages/pluggy/manager.py", line 87, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File ".../site-packages/pluggy/manager.py", line 81, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File ".../site-packages/pluggy/callers.py", line 203, in _multicall
INTERNALERROR>     gen.send(outcome)
INTERNALERROR>   File ".../site-packages/pytest_cov/plugin.py", line 229, in pytest_runtestloop
INTERNALERROR>     self.cov_controller.finish()
INTERNALERROR>   File ".../site-packages/pytest_cov/engine.py", line 171, in finish
INTERNALERROR>     self.cov.stop()
INTERNALERROR>   File ".../site-packages/coverage/control.py", line 575, in save
INTERNALERROR>     data = self.get_data()
INTERNALERROR>   File ".../site-packages/coverage/control.py", line 629, in get_data
INTERNALERROR>     if self._collector and self._collector.flush_data():
INTERNALERROR>   File ".../site-packages/coverage/collector.py", line 425, in flush_data
INTERNALERROR>     self.covdata.add_lines(abs_file_dict(self.data))
INTERNALERROR>   File ".../site-packages/coverage/sqldata.py", line 320, in add_lines
INTERNALERROR>     self._choose_lines_or_arcs(lines=True)
INTERNALERROR>   File ".../site-packages/coverage/sqldata.py", line 368, in _choose_lines_or_arcs
INTERNALERROR>     with self._connect() as con:
INTERNALERROR>   File ".../site-packages/coverage/sqldata.py", line 236, in _connect
INTERNALERROR>     self._create_db()
INTERNALERROR>   File ".../site-packages/coverage/sqldata.py", line 192, in _create_db
INTERNALERROR>     with self._dbs[get_thread_id()] as db:
INTERNALERROR>   File ".../site-packages/coverage/sqldata.py", line 778, in __enter__
INTERNALERROR>     self.connect()
INTERNALERROR>   File ".../site-packages/coverage/sqldata.py", line 756, in connect
INTERNALERROR>     filename = os.path.relpath(self.filename)
INTERNALERROR>   File ".../lib/python3.7/posixpath.py", line 475, in relpath
INTERNALERROR>     start_list = [x for x in abspath(start).split(sep) if x]
INTERNALERROR>   File ".../lib/python3.7/posixpath.py", line 383, in abspath
INTERNALERROR>     cwd = os.getcwd()
INTERNALERROR> FileNotFoundError: [Errno 2] No such file or directory

[simonw]

Maybe this is a bug in pytest-cov now?

[simonw]

Aha - maybe it's this bug over there: pytest-dev/pytest-cov#306