Skip to content

tests/async_fixtures/test_async_gen_fixtures.py::test_async_gen_fixture_finalized is flaky (in 1.0.0a1) #1114

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
mgorny opened this issue May 10, 2025 · 3 comments · Fixed by #1126
Closed

tests/async_fixtures/test_async_gen_fixtures.py::test_async_gen_fixture_finalized is flaky (in 1.0.0a1) #1114

mgorny opened this issue May 10, 2025 · 3 comments · Fixed by #1126
Labels
good first issue
Milestone
v1.0

Comments

@mgorny
Copy link
Contributor

mgorny commented May 10, 2025

While testing 1.0.0a1 on Gentoo, I've noticed that the following test is flaky:

========================================================= test session starts =========================================================
platform linux -- Python 3.11.12, pytest-8.3.5, pluggy-1.5.0 -- /tmp/portage/dev-python/pytest-asyncio-1.0.0_alpha1/work/pytest_asyncio
-1.0.0a1-python3_11/install/usr/bin/python3.11
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase(PosixPath('/tmp/portage/dev-python/pytest-asyncio-1.0.0_alpha1/w
ork/pytest_asyncio-1.0.0a1/.hypothesis/examples'))
rootdir: /tmp/portage/dev-python/pytest-asyncio-1.0.0_alpha1/work/pytest_asyncio-1.0.0a1
configfile: pyproject.toml
testpaths: docs, tests
plugins: xdist-3.6.1
asyncio: mode=Mode.AUTO, asyncio_default_fixture_loop_scope=function, asyncio_default_test_loop_scope=function
created: 12/12 workers
12 workers [166 items]

[…]
============================================================== FAILURES ===============================================================
__________________________________________________ test_async_gen_fixture_finalized ___________________________________________________
[gw0] linux -- Python 3.11.12 /tmp/portage/dev-python/pytest-asyncio-1.0.0_alpha1/work/pytest_asyncio-1.0.0a1-python3_11/install/usr/bi
n/python3.11
tests/async_fixtures/test_async_gen_fixtures.py:37: in test_async_gen_fixture_finalized
    assert mock.called
E   AssertionError: assert False
E    +  where False = <Mock id='139781318067472'>.called
        mock       = <Mock id='139781318067472'>
======================================================= short test summary info =======================================================
FAILED tests/async_fixtures/test_async_gen_fixtures.py::test_async_gen_fixture_finalized - AssertionError: assert False
 +  where False = <Mock id='139781318067472'>.called

It failed for me (on a busy system) roughly 1 out of 3 attempts, once in Python 3.11, twice on 3.13 freethreading.
@mgorny
Copy link
Contributor Author

mgorny commented May 10, 2025

After adding --reruns locally, it fails reliably within the same session — that might suggest that it depends on the order in which tests are run.

@seifertm seifertm added this to the v1.0 milestone May 14, 2025
@seifertm
Copy link
Contributor

Thanks for the report! Historically, the tests weren't isolated against each other. Due to the nature of the tests, they modify global state. This combination turned out to lead to extremely hard-to-debug issues, so we moved to pytest.Testdir and later pytest.Pytester. However, not all of the tests have been ported, yet.

The mentioned test should be wrapped to use Pytester and adjusted if necessary. This needs to happen as part of the v1.0.0 release.

@seifertm seifertm mentioned this issue May 23, 2025
Merged
@seifertm
Copy link
Contributor

@mgorny The test was redundant and was removed in v1.0.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
good first issue
Projects
None yet
Milestone
v1.0
Development

Successfully merging a pull request may close this issue.

Remove obsolete test for async fixtures seifertm/pytest-asyncio
2 participants
@mgorny @seifertm