ENH: mechanism for failing when warnings are issued without conversion to exceptions?

[tylerjereddy]

Maybe this is rather niche, or more of a usage query, but I think it has confused us a little bit. One conundrum we occasionally face in SciPy is that the heavily-used NumPy library sometimes decides to issue warnings for some function calls, but the same code paths can also have try/except guards for various reasons.

So, when we try to proactively turn warnings into errors in filterwarnings in our pytest.ini, what we end up with is a case where a warning becomes invisible to our testsuite/CI testing, and we miss out on i.e., deprecation warnings by the very mechanism we are trying to use to discover them, because the try/except block is triggered via pytest and then handled gracefully by NumPy to end up in a warning-free scenario!

Do you think there's a valid feature request in here? Or should we just resign ourselves to i.e., a CI job that runs the testsuite with the "warnings->errors" disabled and then greps on the pytest summary and errors out on non-zero warnings accounting?

Related:

• DEP: Ensure the __float__ DeprecationWarning is not eaten in np.float64() numpy/numpy#24378
• BUG: NumPyDeprecationWarnings triggered by SciPy without causing test failures scipy/scipy#18781

[RonnyPfannschmidt]

This is a dupe, let me find the one I made

[RonnyPfannschmidt]

#10098 is not quite a dupe but similar enough

I propose post test warnings failure as a feature

[Zac-HD]

I'd consider this a bug in Numpy: the code described in scipy/scipy#18781 (comment) is going to give the wrong answer if users have warnings-as-errors, and for that matter if they try to ctrl-c it'll misbehave badly too. Instead of

try:
   return np.float64(float(obj))
except:
   return np.array(obj, dtype=np.float64)[()]

you should do something like

try:
   return np.float64(float(obj))
except Warning:
    raise
except Exception:
   return np.array(obj, dtype=np.float64)[()]