Summary
Filtering currently works in a basic form, where if any element in a test is selected, the whole test is discarded.
In many cases, it would be much more preferable to exclude specific data points from the metric calculated instead. This requires a few additional changes:
- Analysis most likely should be rewritten in a form where the metric dictionary can be returned as a function callable outside of
pytest (so so reducing dependencies on filters)
- Applications will need custom filter functions, which will require processing of data to pass to the metric getter function. For example, this could be a masked list of errors from data taken from the app's scatter plot.
This could become quite expensive to run.
Example
No response
Alternatives
No response
Additional context
No response
Summary
Filtering currently works in a basic form, where if any element in a test is selected, the whole test is discarded.
In many cases, it would be much more preferable to exclude specific data points from the metric calculated instead. This requires a few additional changes:
pytest(so so reducing dependencies on filters)This could become quite expensive to run.
Example
No response
Alternatives
No response
Additional context
No response