Skip to content

use ParamSpec for pytest.skip instead of __call__ Protocol attribute #13445

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

karlicoss
Copy link

This is a more canonical way of typing generic callbacks/decorators
(see https://mypy.readthedocs.io/en/stable/protocols.html#callback-protocols)

This helps with potential issues/ambiguity with __call__ semanics in type checkers -- according to python spec, __call__ needs to be present on the class, rather than as an instance attribute to be considered callable.

This worked in mypy because it didn't handle the spec 100% correctly (in this case this has no negative consequences)

However, ty type checker is stricter/more correct about it and every pytest.skip usage results in:

error[call-non-callable]: Object of type _WithException[Unknown, <class 'Skipped'>] is not callable

For more context, see:

Testing:

Tested with running mypy against the following snippet:

import pytest
reveal_type(pytest.skip)
reveal_type(pytest.skip.Exception)
reveal_type(pytest.skip(reason="whatever"))

Before the change:

test_pytest_skip.py:2: note: Revealed type is "_pytest.outcomes._WithException[def (reason: builtins.str =, *, allow_module_level: builtins.bool =) -> Never, def (msg: Union[builtins.str, None] =, pytrace: builtins.bool =, allow_module_level: builtins.bool =, *, _use_item_location: builtins.bool =) -> _pytest.outcomes.Skipped]"
test_pytest_skip.py:3: note: Revealed type is "def (msg: Union[builtins.str, None] =, pytrace: builtins.bool =, allow_module_level: builtins.bool =, *, _use_item_location: builtins.bool =) -> _pytest.outcomes.Skipped"
test_pytest_skip.py:4: note: Revealed type is "Never"

After the change:

test_pytest_skip.py:2: note: Revealed type is "_pytest.outcomes._WithException[[reason: builtins.str =, *, allow_module_level: builtins.bool =], Never, def (msg: Union[builtins.str, None] =, pytrace: builtins.bool =, allow_module_level: builtins.bool =, *, _use_item_location: builtins.bool =) -> _pytest.outcomes.Skipped]"
test_pytest_skip.py:3: note: Revealed type is "def (msg: Union[builtins.str, None] =, pytrace: builtins.bool =, allow_module_level: builtins.bool =, *, _use_item_location: builtins.bool =) -> _pytest.outcomes.Skipped"
test_pytest_skip.py:4: note: Revealed type is "Never"

All types are matching and propagated correctly.

This is a more canonical way of typing generic callbacks/decorators
 (see https://mypy.readthedocs.io/en/stable/protocols.html#callback-protocols)

This helps with potential issues/ambiguity with `__call__` semanics in
type checkers -- according to python spec, `__call__` needs to be present
on the class, rather than as an instance attribute to be considered callable.

This worked in mypy because it didn't handle the spec 100% correctly (in this case this has no negative consequences)

However, `ty` type checker is stricter/more correct about it and every `pytest.skip` usage results in:

`error[call-non-callable]: Object of type `_WithException[Unknown, <class 'Skipped'>]` is not callable`

For more context, see:
- astral-sh/ruff#17832 (comment)
- https://discuss.python.org/t/when-should-we-assume-callable-types-are-method-descriptors/92938

Testing:

Tested with running mypy against the following snippet:
```
import pytest
reveal_type(pytest.skip)
reveal_type(pytest.skip.Exception)
reveal_type(pytest.skip(reason="whatever"))
```

Before the change:

```
test_pytest_skip.py:2: note: Revealed type is "_pytest.outcomes._WithException[def (reason: builtins.str =, *, allow_module_level: builtins.bool =) -> Never, def (msg: Union[builtins.str, None] =, pytrace: builtins.bool =, allow_module_level: builtins.bool =, *, _use_item_location: builtins.bool =) -> _pytest.outcomes.Skipped]"
test_pytest_skip.py:3: note: Revealed type is "def (msg: Union[builtins.str, None] =, pytrace: builtins.bool =, allow_module_level: builtins.bool =, *, _use_item_location: builtins.bool =) -> _pytest.outcomes.Skipped"
test_pytest_skip.py:4: note: Revealed type is "Never"
```

After the change:
```
test_pytest_skip.py:2: note: Revealed type is "_pytest.outcomes._WithException[[reason: builtins.str =, *, allow_module_level: builtins.bool =], Never, def (msg: Union[builtins.str, None] =, pytrace: builtins.bool =, allow_module_level: builtins.bool =, *, _use_item_location: builtins.bool =) -> _pytest.outcomes.Skipped]"
test_pytest_skip.py:3: note: Revealed type is "def (msg: Union[builtins.str, None] =, pytrace: builtins.bool =, allow_module_level: builtins.bool =, *, _use_item_location: builtins.bool =) -> _pytest.outcomes.Skipped"
test_pytest_skip.py:4: note: Revealed type is "Never"
```

All types are matching and propagated correctly.
@karlicoss karlicoss force-pushed the pytest_skip_paramspec branch from a0735e1 to 58892a3 Compare May 28, 2025 01:40
@bluetech
Copy link
Member

Thanks! This looks good to me. But looking at this again, I wonder if it wouldn't be simpler (in terms of complexity, not in terms of lines of code) to turn these functions into callable classes? Then the Exception is a regular (class) attribute and there should be no typing shenanigans necessary, unless I'm missing something. WDYT?

@bluetech
Copy link
Member

Forgot to mention, the CI failures are unrelated and fixed in main -- rebase should take care of it.

@karlicoss
Copy link
Author

karlicoss commented Jun 9, 2025

Hi @bluetech -- sorry, just got around to try this, did on a separate branch (just on skip to start with) 56b3937

Something like this (If I understood your suggestion correctly)

+class _Skip:
+    Exception = Skipped
+
+    def __call__(self, reason: str = "", allow_module_level: bool = False) -> NoReturn:
+        __tracebackhide__ = True
+        raise Skipped(msg=reason, allow_module_level=allow_module_level)
+
+skip = _Skip()

This generally works, and seems to result in correct runtime/typecheck time types. However it fails this test:

def test_skip_simple(self):
with pytest.raises(pytest.skip.Exception) as excinfo:
pytest.skip("xxx")
assert excinfo.traceback[-1].frame.code.name == "skip"
assert excinfo.traceback[-1].ishidden(excinfo)
assert excinfo.traceback[-2].frame.code.name == "test_skip_simple"
assert not excinfo.traceback[-2].ishidden(excinfo)

$ tox -e py39-xdist
    def test_skip_simple(self):
        with pytest.raises(pytest.skip.Exception) as excinfo:
            pytest.skip("xxx")
>       assert excinfo.traceback[-1].frame.code.name == "skip"
E       AssertionError: assert '__call__' == 'skip'
E
E         - skip
E         + __call__

testing/python/collect.py:1078: AssertionError

Which kinda makes sense -- python would normally print just in __call__ in the stacktrace, not even in <Classname>.__call__.

Apart from that, the remaining asserts pass if I change the assert to assert excinfo.traceback[-1].frame.code.name == "__call__".
Not sure what's the intent of assert -- if it's just meant to check that skip was called, or we actually want the user to see skip in the trace.

So up to you if you'd prefer me to convert the rest (exit/fail/xfail), move over docstrings and fix the tests; or you'd rather not touch it!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants