only_rerun does not match exceptions wrapped by another exception (__cause__ / __context__ not inspected)

[albertvillanova]

Summary

--only-rerun / only_rerun matches against the outermost exception only. When a transient error is wrapped by another exception via raise ... from error, the pattern can no longer match it, so the test is not retried even though the underlying error is exactly the kind users configure only_rerun for.

Current behaviour

_try_match_error builds the string to match from excinfo alone (src/pytest_rerunfailures.py, master):

def _try_match_error(rerun_errors, excinfo):
    if excinfo:
        err = f"{excinfo.type.__name__}: {excinfo.value}"
        for rerun_error in rerun_errors:
            if isinstance(rerun_error, type) and issubclass(rerun_error, BaseException):
                if issubclass(excinfo.type, rerun_error):
                    return True
            elif re.search(rerun_error, err):
                return True

excinfo.type and excinfo.value are the outermost exception. Neither __cause__ nor __context__ is consulted, so both the regex form and the exception-type form miss a wrapped error.

Reproduction

import pytest


def test_wrapped():
    try:
        raise MemoryError("out of memory")
    except MemoryError as error:
        raise RuntimeError("something failed, see the exception above") from error

$ pytest test_wrapped.py --reruns 2 --only-rerun MemoryError -rR
1 failed in 0.01s          # not rerun

Removing the wrapper makes it retry as expected:

$ pytest test_direct.py --reruns 2 --only-rerun MemoryError -rR
1 failed, 2 rerun in 0.02s

Why this matters in practice

We hit this in the TRL CI, where the suite runs under pytest-xdist on a shared GPU and --only-rerun includes OutOfMemoryError to absorb memory pressure between workers.

When the OOM is raised inside torch.testing.assert_close, torch.testing wraps any exception it does not expect:

raise RuntimeError(
    f"Comparing\n\n"
    f"{pair}\n\n"
    f"resulted in the unexpected exception above. "
    ...
) from error

The failure then surfaces as RuntimeError: Comparing ..., with torch.OutOfMemoryError demoted to __cause__. The wrapper message does not carry the original error text, so no only_rerun value can match it: neither OutOfMemoryError (wrong outer type) nor CUDA out of memory (absent from the outer message).

The only workaround available to us is matching the wrapper text itself, which we rejected: torch.testing emits that message for any unexpected exception during a comparison, so it would also retry genuine defects. A deterministic bug would still fail all attempts, but a nondeterministic one would be silently retried into green, which is precisely what a narrow only_rerun is meant to prevent.

Proposed behaviour

Walk the exception chain and match if any linked exception matches, something like:

def _iter_exception_chain(exc):
    seen = set()
    while exc is not None and id(exc) not in seen:
        seen.add(id(exc))
        yield exc
        exc = exc.__cause__ or exc.__context__


def _try_match_error(rerun_errors, excinfo):
    if not excinfo:
        return False
    for exc in _iter_exception_chain(excinfo.value):
        err = f"{type(exc).__name__}: {exc}"
        for rerun_error in rerun_errors:
            if isinstance(rerun_error, type) and issubclass(rerun_error, BaseException):
                if isinstance(exc, rerun_error):
                    return True
            elif re.search(rerun_error, err):
                return True
    return False

With that, the existing OutOfMemoryError pattern matches the wrapped case on its own, with no ambiguity and no wrapper-text heuristic.

Two notes on scope:

• This is unrelated to For "only_rerun" allow access exception attributes #230. That asked for a user-supplied condition callback, whose implementation (Allow to pass a callable condition to the flaky marker #299) was reverted in Make pytest-xdist happy again #304 for pytest-xdist incompatibility. Chain walking needs no user callback and no new serialization: it reads type names and str() of already-available exceptions, and the decision is still taken worker-side in pytest_runtest_makereport, exactly where it is taken today.
• __cause__ is the unambiguous signal (explicit raise ... from). Following __context__ as well also covers exceptions raised inside an except block without from, which is the more common accidental wrapping. If following __context__ is considered too broad, restricting to __cause__ alone would already fix the case above and could be worth gating behind an option.

Happy to open a PR if the approach looks right.

Environment

• pytest-rerunfailures 16.6 (behaviour unchanged on master), also reproduced on 15.1

[albertvillanova]

I validated the proposal by monkeypatching _try_match_error with the implementation above and rerunning the same cases:

# wrapped MemoryError, with the chain-walking match
RERUN test_wrapped.py::test_wrapped
RERUN test_wrapped.py::test_wrapped
1 failed, 2 rerun in 0.04s

# control: an unrelated ValueError, same --only-rerun MemoryError
1 failed in 0.04s

So the wrapped case is retried while an unrelated failure is still failed immediately, with no change to the existing patterns.

I am happy to turn this into a PR, with tests covering a __cause__ chain, a __context__ chain, the exception-type form of only_rerun, and a non-matching control.

[icemac]

Thank you for the issue, a pull request is welcome.