Dynamically calling async fixture causes a runtime error saying "This event loop is already running"

[ykuzma1]

There seems to be a bug with how request.getfixturevalue(argname) interacts with pytest-asyncio. Calling the function leads to a runtime error saying the event loop is already running. If you change it from being dynamically called to being fixed in the function definition, it works as expected.

Platform Info:

platform win32 -- Python 3.7.2, pytest-4.2.0, py-1.7.0, pluggy-0.8.1
plugins: asyncio-0.11.0.dev0

Test fixture that can be used in both dynamic/fixed cases:

@pytest.fixture
async def async_fixture():
    yield 'Hi from async_fixture()!'

Successful fixed function argument fixture test:

@pytest.mark.asyncio
async def test_async_fixture_fixed(async_fixture):
    assert async_fixture == 'Hi from async_fixture()!'

Failed dynamic fixture test:

@pytest.mark.asyncio
async def test_async_fixture_dynamic(request):
    async_fixture = request.getfixturevalue('async_fixture')
    assert async_fixture == 'Hi from async_fixture()!'

Failed test trace-back:

================================== FAILURES ===================================
_________________________ test_async_fixture_dynamic __________________________

request = <FixtureRequest for <Function test_async_fixture_dynamic>>

    @pytest.mark.asyncio
    async def test_async_fixture_dynamic(request):
>       async_fixture = request.getfixturevalue('async_fixture')

tests\test_app_factory.py:52: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
venv\lib\site-packages\_pytest\fixtures.py:478: in getfixturevalue
    return self._get_active_fixturedef(argname).cached_result[0]
venv\lib\site-packages\_pytest\fixtures.py:501: in _get_active_fixturedef
    self._compute_fixture_value(fixturedef)
venv\lib\site-packages\_pytest\fixtures.py:586: in _compute_fixture_value
    fixturedef.execute(request=subrequest)
venv\lib\site-packages\_pytest\fixtures.py:881: in execute
    return hook.pytest_fixture_setup(fixturedef=self, request=request)
venv\lib\site-packages\pluggy\hooks.py:284: in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
venv\lib\site-packages\pluggy\manager.py:68: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
venv\lib\site-packages\pluggy\manager.py:62: in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
venv\lib\site-packages\_pytest\fixtures.py:923: in pytest_fixture_setup
    result = call_fixture_func(fixturefunc, request, kwargs)
venv\lib\site-packages\_pytest\fixtures.py:782: in call_fixture_func
    res = fixturefunc(**kwargs)
..\pytest-asyncio\pytest_asyncio\plugin.py:97: in wrapper
    return loop.run_until_complete(setup())
C:\Users\ykuzm\AppData\Local\Programs\Python\Python37\lib\asyncio\base_events.py:571: in run_until_complete
    self.run_forever()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <_WindowsSelectorEventLoop running=False closed=False debug=False>

    def run_forever(self):
        """Run until stop() is called."""
        self._check_closed()
        if self.is_running():
>           raise RuntimeError('This event loop is already running')
E           RuntimeError: This event loop is already running

[ykuzma1]

I'd be happy to put in a pull request. I mostly want to use this space for ideas, because I've tried everything I can think of. Ideas welcome! I'll put my couple attempts below.

[ykuzma1]

My thinking is that it fails because plugin.py calls loop.run_until_complete(setup()) during the middle of async tests being run on the event loop already. So calling run_until_complete tries adding the async fixture onto the already running loop - causing the error. It sounds like iPython had the same issue. So I tried some workarounds they suggested to replace that line:

Causes infinite loop:

return asyncio.run_coroutine_threadsafe(setup(), loop).result()

Causes asyncio.base_futures.InvalidStateError: Result is not ready. error:

return asyncio.ensure_future(setup()).result()

Another infinite loop:

from concurrent.futures import ThreadPoolExecutor
loop = asyncio.new_event_loop()
ThreadPoolExecutor().submit(loop.run_forever)
return asyncio.run_coroutine_threadsafe(setup(), loop).result()

[ykuzma1]

Funny enough I got one workaround to work while I was going back through the iPython thread:

from concurrent.futures import ThreadPoolExecutor
pool = ThreadPoolExecutor(1)
loop = asyncio.new_event_loop()
pool.submit(asyncio.set_event_loop, loop).result()
return pool.submit(loop.run_until_complete, setup()).result()

Will start working on a proper pull request.

[senciucserban]

Any news? 😢

[brianmaissy]

any updates on this?

[dazza-codes]

Sorry this doesn't help solve this for pytest-asyncio, but I struggled with this for a while, including trying custom fixtures with module scope and nest_asyncio, but eventually a solution was to drop all the pytest-asyncio decorators and async def tests to go back to vanilla non-async test functions with a non-async entry point that runs the rest of the async coroutines. The only trick to executing the integration tests for the async coroutines was to create a main like entry point that all the tests pass through and that function gets a new event loop every time, e.g.

def run_aync_main(*args, **kwargs):
    main_loop = asyncio.new_event_loop()
    try:
        main_loop.run_until_complete(any_async_entry_point(*args, *kwargs))
    finally:
        main_loop.stop()
        main_loop.close()

Any non-async function that calls a run_until_complete does not need to be run by pytest-asyncio decorators and it should not conflict across tests when it gets a new loop. Other unit tests on async coroutines with regular await statements work OK with pytest-asyncio.

[PidgeyBE]

In our case it worked by simply making the dynamic fixture sync:

def test_async_fixture_dynamic(request, event_loop):
    async_fixture = request.getfixturevalue('async_fixture')
    assert async_fixture == 'Hi from async_fixture()!'

[roganov]

Got this error when I tried to upgrade to 0.14.0 from 0.10.0. Rolled back for now.

[dorindivo1]

Got this error in 0.15.1 also, any update ?

[ReznikovRoman]

got this error in 0.18.3

[seifertm]

As of v0.18.3 this error could be caused by an unexpected interaction with other pytest plugins that manipulate the event loop.

@ReznikovRoman Can you provide a reproducible example?

[vadim-su]

I got the same issue today.

Code

@fixture
async def client():
    async with httpx.AsyncClient() as http_client:
        yield HavenClient(
            http_client,
            TEST_APIKEY,
        )


@fixture
async def client_without_apikey():
    async with httpx.AsyncClient() as http_client:
        yield HavenClient(
            http_client,
        )

@pytest.mark.parametrize('client_fixture,expectation', [
    ('client', do_not_raise()),
    ('client_without_apikey', pytest.raises(errors.ApikeyNotSetError)),
])
async def test_get_user_settings(client_fixture, expectation, request: pytest.FixtureRequest):
    client: HavenClient = request.getfixturevalue(client_fixture)
    with expectation:
        settings = await client.get_user_settings()
        assert settings

    @pytest.mark.parametrize('client_fixture,expectation', [
        ('client', do_not_raise()),
        ('client_without_apikey', pytest.raises(errors.ApikeyNotSetError)),
    ])
    async def test_get_user_settings(client_fixture, expectation, request: pytest.FixtureRequest):
>       client: HavenClient = request.getfixturevalue(client_fixture)

self = <_UnixSelectorEventLoop running=False closed=False debug=False>

    def _check_running(self):
        if self.is_running():
>           raise RuntimeError('This event loop is already running')
E           RuntimeError: This event loop is already running

/usr/local/lib/python3.10/asyncio/base_events.py:582: RuntimeError

Full output

pytest -k test_get_user_settings                     
======================================================================================== test session starts ========================================================================================
platform linux -- Python 3.10.4, pytest-7.1.2, pluggy-1.0.0
rootdir: /workspaces/haven, configfile: pyproject.toml, testpaths: tests
plugins: anyio-3.6.1, asyncio-0.18.3
asyncio: mode=auto
collected 11 items / 9 deselected / 2 selected                                                                                                                                                      

tests/test_client.py FF                                                                                                                                                                       [100%]

============================================================================================= FAILURES ==============================================================================================
____________________________________________________________________________ test_get_user_settings[client-expectation0] ____________________________________________________________________________

client_fixture = 'client', expectation = <contextlib.suppress object at 0x7f33c81ee9b0>, request = <FixtureRequest for <Function test_get_user_settings[client-expectation0]>>

    @pytest.mark.parametrize('client_fixture,expectation', [
        ('client', do_not_raise()),
        ('client_without_apikey', pytest.raises(errors.ApikeyNotSetError)),
    ])
    async def test_get_user_settings(client_fixture, expectation, request: pytest.FixtureRequest):
>       client: HavenClient = request.getfixturevalue(client_fixture)

tests/test_client.py:80: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:554: in getfixturevalue
    fixturedef = self._get_active_fixturedef(argname)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:573: in _get_active_fixturedef
    self._compute_fixture_value(fixturedef)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:659: in _compute_fixture_value
    fixturedef.execute(request=subrequest)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:1057: in execute
    result = ihook.pytest_fixture_setup(fixturedef=self, request=request)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/pluggy/_hooks.py:265: in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/pluggy/_manager.py:80: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:1111: in pytest_fixture_setup
    result = call_fixture_func(fixturefunc, request, kwargs)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:890: in call_fixture_func
    fixture_result = fixturefunc(**kwargs)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/pytest_asyncio/plugin.py:293: in _asyncgen_fixture_wrapper
    result = event_loop.run_until_complete(setup())
/usr/local/lib/python3.10/asyncio/base_events.py:622: in run_until_complete
    self._check_running()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <_UnixSelectorEventLoop running=False closed=False debug=False>

    def _check_running(self):
        if self.is_running():
>           raise RuntimeError('This event loop is already running')
E           RuntimeError: This event loop is already running

/usr/local/lib/python3.10/asyncio/base_events.py:582: RuntimeError
____________________________________________________________________ test_get_user_settings[client_without_apikey-expectation1] _____________________________________________________________________

client_fixture = 'client_without_apikey', expectation = <_pytest.python_api.RaisesContext object at 0x7f33c81ee950>
request = <FixtureRequest for <Function test_get_user_settings[client_without_apikey-expectation1]>>

    @pytest.mark.parametrize('client_fixture,expectation', [
        ('client', do_not_raise()),
        ('client_without_apikey', pytest.raises(errors.ApikeyNotSetError)),
    ])
    async def test_get_user_settings(client_fixture, expectation, request: pytest.FixtureRequest):
>       client: HavenClient = request.getfixturevalue(client_fixture)

tests/test_client.py:80: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:554: in getfixturevalue
    fixturedef = self._get_active_fixturedef(argname)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:573: in _get_active_fixturedef
    self._compute_fixture_value(fixturedef)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:659: in _compute_fixture_value
    fixturedef.execute(request=subrequest)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:1057: in execute
    result = ihook.pytest_fixture_setup(fixturedef=self, request=request)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/pluggy/_hooks.py:265: in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/pluggy/_manager.py:80: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:1111: in pytest_fixture_setup
    result = call_fixture_func(fixturefunc, request, kwargs)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:890: in call_fixture_func
    fixture_result = fixturefunc(**kwargs)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/pytest_asyncio/plugin.py:293: in _asyncgen_fixture_wrapper
    result = event_loop.run_until_complete(setup())
/usr/local/lib/python3.10/asyncio/base_events.py:622: in run_until_complete
    self._check_running()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <_UnixSelectorEventLoop running=False closed=False debug=False>

    def _check_running(self):
        if self.is_running():
>           raise RuntimeError('This event loop is already running')
E           RuntimeError: This event loop is already running

/usr/local/lib/python3.10/asyncio/base_events.py:582: RuntimeError
====================================================================================== short test summary info ======================================================================================
FAILED tests/test_client.py::test_get_user_settings[client-expectation0] - RuntimeError: This event loop is already running
FAILED tests/test_client.py::test_get_user_settings[client_without_apikey-expectation1] - RuntimeError: This event loop is already running
================================================================================== 2 failed, 9 deselected in 0.38s ==================================================================================
sys:1: RuntimeWarning: coroutine '_wrap_asyncgen.<locals>._asyncgen_fixture_wrapper.<locals>.setup' was never awaited
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

I'll try to get a clean example little bit later.