Async fixtures may break current event loop

[MarkusSintonen]

It seems pytest-asyncio (0.23.7) is currently badly broken with code relying on aiohttp.

This simple example currently breaks. The example is heavily simplified from the actual fixture setup. This used to work fine before 0.22.

import aiohttp
import pytest


# No longer working for getting a single event loop for aiohttp
# @pytest.fixture(scope="session")
# def event_loop():
#    loop = asyncio.get_event_loop_policy().new_event_loop()
#    try:
#        yield loop
#    finally:
#        loop.close()


@pytest.fixture(scope="session")
async def client(
    # event_loop <- accessing this no longer working. But neither this works without it
):
    async with aiohttp.ClientSession(
        # loop=event_loop <- aiohttp requires using a single event_loop that can not be closed between the tests...
    ) as session:
        yield session


async def test_the_client(client: aiohttp.ClientSession):
    await client.get("http://localhost:8080/foobar")  # RuntimeError: Timeout context manager should be used inside a task

Using asyncio_mode = "auto":

[tool.poetry]
name = "foobar"
package-mode = false

[tool.poetry.dependencies]
python = "~3.12"
aiohttp = "==3.9.5"

[tool.poetry.dev-dependencies]
pytest = "==8.2.2"
pytest-asyncio = "==0.23.7"

[tool.pytest.ini_options]
asyncio_mode = "auto"

[MarkusSintonen]

Using this https://pytest-asyncio.readthedocs.io/en/latest/how-to-guides/run_session_tests_in_same_loop.html doesnt have any effect. Seems there are multiple issues now in pytest-asyncio. There is no longer access to a single event loop used for tests. Also pytest-asyncio is now closing the event loop between the tests. How do I get back the previous working behaviour which is compatible eg with aiohttp? It is neither possible to change the session scoped client fixture to a function fixture. (Also it doesnt make sense, we want a single client per test session)

[MarkusSintonen]

There seems to be multiple open issues related to latest version of pytest-asyncio. Couldn't find anything directly related to aiohttp usage.

[seifertm]

Thanks for the code example.

In your specific code, the issue is that the test and the fixture run in different asyncio event loops. Each pytest scope has its own event loop. When specifying a @pytest.fixture(scope="session"), the fixture code runs in a session-scoped loop, whereas test_the_client uses the default function-scoped loop.

Adding @pytest.mark.asyncio(scope="session") to test_the_client make the test code to run in the same session-scoped loop, thus eliminating the issue.

However, there's a tightly linked know issue in pytest-asyncio, which makes it impossible to separate the scope of the event loop from the scope of the pytest fixture. Sometimes, it's necessary to have code that runs in a session-wide event loop, but that code should be re-executed for every function. This is not possible with v0.23 at the moment. If you require this functionality, you should pin pytest-asyncio to v0.21 until the next major release. This issue is tracked in #706.

As for why the code mentioned in the docs doesn't work, that's an entirely different topic that needs investigating.

[seifertm]

When I add the following code snippet to conftest.py (instead of @pytest.mark.asyncio(scope="session" as mentioned in my previous comment), your provided example runs as expected.

import pytest

from pytest_asyncio import is_async_test

def pytest_collection_modifyitems(items):
    pytest_asyncio_tests = (item for item in items if is_async_test(item))
    session_scope_marker = pytest.mark.asyncio(scope="session")
    for async_test in pytest_asyncio_tests:
        async_test.add_marker(session_scope_marker, append=False)

I referred to the exact same documentation page as you did (see How to run all tests in the session in the same event loop).

Do you think you could provide another code example where the code snippet in the docs doesn't solve the issues you're experiencing?

[MarkusSintonen]

@seifertm Thanks

It seems I was missing the append=False parameter 🤔 Strange... It partially works with that. Btw could we instead have some config parameter to not have to add this boilerplate to every test suite (like with asyncio_mode toml param)?

But even with this, I still have problems getting it working with more complex sets of fixtures. I was able to narrow the problems into a next weird case. It again breaks with RuntimeError: Timeout context manager should be used inside a task when adding an async-autouse fixture (to either to the test file or to the conftest):

@pytest.fixture(autouse=True)
async def some_function_scope_autouse_async_fixture():
    pass

Full failing example:

import aiohttp
import pytest


@pytest.fixture(autouse=True)
async def some_function_scope_autouse_async_fixture():
    pass


@pytest.fixture(scope="session")
async def client():
    async with aiohttp.ClientSession() as session:
        yield session


async def test_the_client(client):
    async with client.get("http://localhost:8080/foobar") as resp:  # RuntimeError: Timeout context manager should be used inside a task
        resp.raise_for_status()

The pytest.mark.asyncio(scope="session")-thing in conftest doesnt help here... In real usage Im also seeing Event loop is closed errors. But I wasnt yet able to narrow what exactly causes that other error yet. Lets first investigate the above issue with the autouse fixture...

[seifertm]

Thanks for the reproducer!

The autouse fixture seems to pull in the function-scoped event_loop fixture as a dependency of test_the_client in addition to the session-scoped loop. I can verify this using pytest's --setup-show output:

$ pytest --asyncio-mode=auto --setup-show
===== test session starts =====
platform linux -- Python 3.12.4, pytest-8.2.2, pluggy-1.5.0
rootdir: /tmp/tst
plugins: asyncio-0.23.7
asyncio: mode=Mode.AUTO
collected 1 item                                                                                                                                                                                                                                                                                                    

test_a.py 
SETUP    S event_loop_policy
SETUP    S _session_event_loop (fixtures used: event_loop_policy)
SETUP    S client (fixtures used: _session_event_loop)
        SETUP    F event_loop
        SETUP    F some_function_scope_autouse_async_fixture (fixtures used: event_loop)
        test_a.py::test_the_client (fixtures used: _session_event_loop, client, event_loop, event_loop_policy, request, some_function_scope_autouse_async_fixture)F
        TEARDOWN F some_function_scope_autouse_async_fixture
        TEARDOWN F event_loop
TEARDOWN S client
TEARDOWN S _session_event_loop
TEARDOWN S event_loop_policy

===== FAILURES =====
_____ test_the_client _____

client = <aiohttp.client.ClientSession object at 0x7fa285fbf3b0>

    @pytest.mark.asyncio(scope="session")
    async def test_the_client(client):
>       async with client.get("http://localhost:8080/foobar") as resp:  # RuntimeError: Timeout context manager should be used inside a task

test_a.py:18: 
_ _ _
venv/lib/python3.12/site-packages/aiohttp/client.py:1197: in __aenter__
    self._resp = await self._coro
venv/lib/python3.12/site-packages/aiohttp/client.py:507: in _request
    with timer:
_ _ _

self = <aiohttp.helpers.TimerContext object at 0x7fa285fbcc80>

    def __enter__(self) -> BaseTimerContext:
        task = current_task(loop=self._loop)
    
        if task is None:
>           raise RuntimeError(
                "Timeout context manager should be used " "inside a task"
            )
E           RuntimeError: Timeout context manager should be used inside a task

venv/lib/python3.12/site-packages/aiohttp/helpers.py:715: RuntimeError
===== short test summary info =====
FAILED test_a.py::test_the_client - RuntimeError: Timeout context manager should be used inside a task
===== 1 failed in 0.12s =====

Since the event_loop fixture is evaluated after the setup of the session-scoped client fixture, the test ends up running in the event_loop loop.

This should actually result in a MultipleEventLoopsRequestedError, but it seems to be circumvented by the autouse mechanism. This is clearly a bug in pytest-asyncio.

Until this is fixed, you can change the scope of some_function_scope_autouse_async_fixture to scope="session", if your test suite allows that.

[seifertm]

It seems I was missing the append=False parameter 🤔 Strange... It partially works with that. Btw could we instead have some config parameter to not have to add this boilerplate to every test suite (like with asyncio_mode toml param)?

This feature has been requested a number of times and it's very reasonable. There are plans to add this kind of "default event loop scope" in #871

[MarkusSintonen]

Until this is fixed, you can change the scope of some_function_scope_autouse_async_fixture to scope="session", if your test suite allows that.

No that's not possible and won't make sense in the real usage. 😕 The example was greatly simplified.