Breaking change in 0.23.*

[tkukushkin]

Hello! Something has been broken with the latest pytest-asyncio releases.

Consider such code:

import asyncio
import pytest

@pytest.fixture(scope='session')
def event_loop():
    loop = asyncio.get_event_loop_policy().new_event_loop()
    yield loop
    loop.close()

@pytest.fixture(scope='session')
async def some_async_fixture():
    return asyncio.get_running_loop()

async def test_something(some_async_fixture):
    assert asyncio.get_running_loop() is some_async_fixture

pytest.ini:

[pytest]
asyncio_mode = auto

This test passes with pytest-asyncio 0.21.1 but fails with 0.23.0. I am not sure if it's ok, but if it is, IMHO it is breaking change.

[redtomato74]

I confirm, we use similar setup with 2 session level fixtures (one to redefine event loop, another for our own purposes), tests don't work anymore, complain either about "The future belongs to a different loop than the one specified as the loop argument" or "Event loop is closed".

[albertferras-vrf]

The version 0.23.0 changelog is already mentioning the breaking change: https://github.com/pytest-dev/pytest-asyncio/releases/tag/v0.23.0

I went through the same, the new way to do it can be seen in this PR https://github.com/pytest-dev/pytest-asyncio/pull/662/files
Would be nice to have some documentation on to how to migrate to the new way.

[redtomato74]

It says "This release is backwards-compatible with v0.21. Changes are non-breaking, unless you upgrade from v0.22."

[tkukushkin]

@albertferras-vrf the changelog mentions asyncio_event_loop mark removal, I think it is only about upgrading from 0.22.

[albertferras-vrf]

You are right, I forgot about that part

[seifertm]

It says "This release is backwards-compatible with v0.21. Changes are non-breaking, unless you upgrade from v0.22."

That was the original intention, yes. I can reproduce the difference between v0.21.1 and v0.23 and I agree that this is a breaking change (by accident).

With regards to the migration and as a workaround: The fundamental idea of v0.23 is that each pytest scope (session, package, module, class, and function) provides a separate event loop. You can decide for each test in which loop they run via the new scope kwarg to the asyncio mark.

@tkukushkin In your specific example you want to run the test in the same loop as the fixture. The some_async_fixture fixture has session scope. In order to achieve your goal, you should mark test_something accordingly:

@pytest.mark.asyncio(scope="session")
async def test_something(some_async_fixture):
    assert asyncio.get_running_loop() is some_async_fixture

See also the part about asyncio event loops in the Concepts section of the docs.

@redtomato74 I'd like to hear more about your use case for two different event loops. I suggest you open a separate issue for this.

[tkukushkin]

@seifertm I'd like to have only one loop for all fixtures and tests, without additional decorators to all tests and fixtures. We have thousands of tests in hundreds of services where all tests and fixtures share one loop and it is crucial for them. Is there any workaround to emulate the old behaviour?

This way https://pytest-asyncio.readthedocs.io/en/v0.23.2/how-to-guides/run_session_tests_in_same_loop.html does not consider fixtures at all(

[seifertm]

I'd like to have only one loop for all fixtures and tests, without additional decorators to all tests and fixtures.

The linked how-to is supposed to make it easy to add the asyncio mark to all of your tests, specifically for large test suites where marking all packages or modules is cumbersome. Unfortunately, there seem to be problems with it, too. Please refer to #705

We have thousands of tests in hundreds of services where all tests and fixtures share one loop and it is crucial for them.

I don't think this use case was considered during the development of v0.22 and v0.23. Can you explain why you need all tests and fixtures to run the same loop? Why

This way https://pytest-asyncio.readthedocs.io/en/v0.23.2/how-to-guides/run_session_tests_in_same_loop.html does not consider fixtures at all(
That's a fair point. The docs should be updated to explain how fixtures are run.

Fixtures in v0.23 generally behave like tests and choose the event loop of the fixture scope. That means if a fixture has session scope it will run in the session-wide loop.

I cannot think of a workaround to switch to the old behaviour at the moment. I suggest pinning pytest-asyncio to <0.23 until this issue is fixed.

[tkukushkin]

Fixtures in v0.23 generally behave like tests and choose the event loop of the fixture scope. That means if a fixture has session scope it will run in the session-wide loop.

Sorry, but I don't understand, I have not only session scoped fixtures but also module scoped fixtures, and they should use the same event loop as session scoped fixtures. Could you please describe how to achieve it?

We have thousands of tests in hundreds of services where all tests and fixtures share one loop and it is crucial for them.

I don't think this use case was considered during the development of v0.22 and v0.23. Can you explain why you need all tests and fixtures to run the same loop?

We write blackbox tests for microservices using pytest and pytest-asyncio. Some session scoped fixtures for example create database connection pool, which all tests can use to check database state. Another session scoped fixture in background monitors logs of subprocesses (instances of application, that we test) and captures these logs to some list, which tests can check. These subprocesses can be started by any fixture and test as async context manager. And obviously, subprocess (asyncio.subprocess) should be started with the same loop as fixture that captures logs from it. And we have way more such examples.

[seifertm]

Thanks for the explanation!

Fixtures in v0.23 generally behave like tests and choose the event loop of the fixture scope. That means if a fixture has session scope it will run in the session-wide loop.

Sorry, but I don't understand, I have not only session scoped fixtures but also module scoped fixtures, and they should use the same event loop as session scoped fixtures. Could you please describe how to achieve it?

This is what I meant when I said your use case hasn't been considered in the development. There's currently no way to control the event loop used by a fixture independently from the fixture scope.

The v0.23 release will not work for your test suite. I suggest that you downgrade to v0.21.1.

I'll think of a way to control the fixture scope independently of the event loop scope.

[tkukushkin]

Yes, we have already downgraded to 0.21.1. I don't think it's gonna be a problem for us for a long time (at least until Python 3.13).

I'll think of a way to control the fixture scope independently of the event loop scope.

Thank you! Looking forward to the news.