got Future <Future pending> attached to a different loop

[kaijajan]

I'm working on a pytest/pytest-asyncio/asyncio test, however, I encounterd a strange issue "got Future attached to a different loop"
Not quite sure if my issue is exactly the same as others.
I created client/server tests using pytest and asyncio, the fixture would return a client object contains asyncio streams object.
But while multiple tests executed in tests, the error "got Future attached to a different loop" shows. (the test would pass if only one test func in script)
And have been studying for a while and have no clue how to move on.
Any idea which step went wrong? and Anyone knows how to solve the issue?
Thanks

my test code:
https://github.com/kaijajan/pytest_asyncio_issue

The following block is the log.

❯ pytest -v --capture=no
=========================================== test session starts ============================================
platform darwin -- Python 3.7.7, pytest-5.4.3, py-1.8.1, pluggy-0.13.1 -- /usr/local/opt/python/bin/python3.7
cachedir: .pytest_cache
metadata: {'Python': '3.7.7', 'Platform': 'Darwin-18.6.0-x86_64-i386-64bit', 'Packages': {'pytest': '5.4.3', 'py': '1.8.1', 'pluggy': '0.13.1'}, 'Plugins': {'timeout': '1.3.4', 'metadata': '1.9.0', 'asyncio': '0.12.0', 'html': '2.1.1'}}
rootdir: /Volumes/MacData/workspaces/python/temp/pytest_asyncio_issue
plugins: timeout-1.3.4, metadata-1.9.0, asyncio-0.12.0, html-2.1.1
collected 2 items                                                                                          

test_1.py::test_1 client init
connecting
connected, yielding
received: hello 0 returned
received: hello 1 returned
received: hello 2 returned
received: hello 3 returned
received: hello 4 returned
received: hello 5 returned
received: hello 6 returned
received: hello 7 returned
received: hello 8 returned
received: hello 9 returned
hello world
PASSED
test_1.py::test_2 FAILED

================================================= FAILURES =================================================
__________________________________________________ test_2 __________________________________________________

client = <client.SimpleClient object at 0x10dd3b610>

    @pytest.mark.usefixtures('client')
    @pytest.mark.asyncio
    async def test_2(client):
        for i in range(10):
            await client.send(f'hello {i}')
>           data = await client.receive()

test_1.py:21: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
client.py:25: in receive
    tmp = await self.reader.read(size)
/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/streams.py:640: in read
    await self._wait_for_data('read')
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <StreamReader transport=<_SelectorSocketTransport fd=17>>, func_name = 'read'

    async def _wait_for_data(self, func_name):
        """Wait until feed_data() or feed_eof() is called.
    
        If stream was paused, automatically resume it.
        """
        # StreamReader uses a future to link the protocol feed_data() method
        # to a read coroutine. Running two read coroutines at the same time
        # would have an unexpected behaviour. It would not possible to know
        # which coroutine would get the next data.
        if self._waiter is not None:
            raise RuntimeError(
                f'{func_name}() called while another coroutine is '
                f'already waiting for incoming data')
    
        assert not self._eof, '_wait_for_data after EOF'
    
        # Waiting for data while paused will make deadlock, so prevent it.
        # This is essential for readexactly(n) for case when n > self._limit.
        if self._paused:
            self._paused = False
            self._transport.resume_reading()
    
        self._waiter = self._loop.create_future()
        try:
>           await self._waiter
E           RuntimeError: Task <Task pending coro=<test_2() running at /Volumes/MacData/workspaces/python/temp/pytest_asyncio_issue/test_1.py:21> cb=[_run_until_complete_cb() at /usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py:157]> got Future <Future pending> attached to a different loop

/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/streams.py:473: RuntimeError
========================================= short test summary info ==========================================
FAILED test_1.py::test_2 - RuntimeError: Task <Task pending coro=<test_2() running at /Volumes/MacData/wo...
======================================= 1 failed, 1 passed in 1.13s ========================================

[alblasco]

Your error message (attached to a different loop) seems similar to previous #162 that I reported.
And my PR#164 fixing #162 has recently been merged in master branch, but a new version has not yet been released.
So I recommend you to try current master branch code, to see if it solves your issue too.
You can just copy-paste plugin.py in your installed pytest-asyncio.

Please let us know your feedback

EDIT 1:

Nevertheless, I have provided a PR with some additional changes in your pytest_asyncio_issue code.

See as reference:

• Streams
• Fixture finalization

EDIT 2:

Only with the suggested changes in your code, see PR, it works fine with current pytest-asyncio V0.12.0.
So it seems not to be a pytest-asyncio problem.
Note: Tested in windows with open_connection/start_server, instead of open_unix_connection/start_unix_server

[Tinche]

I had a hellish two weeks at work so the release got pushed back, I'll try to release today.

[seifertm]

This issue was reported for pytest-asyncio-0.12.0. As far as I understand, the problem has been solved. If not, please retest with the latest version of pytest-asyncio and reopen the issue or file a new one.