Reference documentation, typing annotations and reality for `pytest.mark.xfail()` don’t match

[manueljacob]

The documentation for pytest.mark.xfail() does not match reality.

• reason=None: If no conditions are passed, reason’s default is "" (code). Adding @pytest.mark.xfail(reason=None, strict=True) to a passing test results in “TypeError: can only concatenate str (not "NoneType") to str”. If a condition is passed, passing no reason behaves like passing None to it (code). I’m unsure how this subtlety could be captured in the function signature. pytest.mark.skipif() has the same issue. pytest.mark.skip()’s default value for reason is in fact "unconditional skip" (code).
• raises: The type (according to the type annotations) is Union[Type[BaseException], Tuple[Type[BaseException], ...]] instead of Type[Exception]. Neither captures the fact that None can be passed to it (which is also the documented default value).
• strict=False: The default value of strict is actually determined by the xfail_strict config (code).

[TanyaAgarwal28]

Hi @Zac-HD I would like to pick this up

[Zac-HD]

@TanyaAgarwal28 you don't need permission to open a PR, just go ahead.

Like most maintainers, I'm a volunteer and prefer not to be pinged on issues or PRs - I'm already subscribed to everything I want notifications about. Someone will review your PRs when they have free time; and if they don't it could be a while. That's just how volunteer projects go!

[Avasam]

Allowing condition=None (as show in the current doc method signature) can also reduce overly verbose redundant code. See this example from setuptools:

    @pytest.mark.xfail(
        # Argument of type "str | None" cannot be assigned to parameter "condition" of type "str | bool" in function "__call__" Pylance[reportGeneralTypeIssues]
        # Argument 1 to "__call__" of "_XfailMarkDecorator" has incompatible type "Optional[str]"; expected "Union[str, bool]" Mypy[arg-type]
        os.environ.get('PYTHONDONTWRITEBYTECODE'),
        reason="Byte code disabled",
    )
    def test_exclude_source_files(self, setup_context, user_override): ...

Where the current workaround is os.environ.get('PYTHONDONTWRITEBYTECODE', False)