Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Image-based plot testing #93

Open
ajjackson opened this issue Sep 7, 2020 · 0 comments
Open

Image-based plot testing #93

ajjackson opened this issue Sep 7, 2020 · 0 comments
Labels
nice to have Not urgently required, but would be an improvement

Comments

@ajjackson
Copy link
Collaborator

At present:

  • the dispersion.py and dos.py scripts are tested by grabbing the generated figure using plt.gcf() and inspecting ax.lines
  • There are legacy tests for euphonic.plot which inspect tick labels and ax.lines for plot_1d and plot_dispersion
  • parts of euphonic.plot are untested (i.e. plot_2d)

While I have advocated for this kind of test in the past (they are robust and verify that numbers are not changing unexpectedly), they are cumbersome to set up and can miss more general issues with plot configuration.

It could be interesting to try https://github.com/matplotlib/pytest-mpl which provides a pytest fixture for comparing output images with reference results. This seems a robust sanity check against surprises, which also provides a visual output to quickly inspect differences. I've been cautious in the past about image checks due to the potential fragility against minor differences in Matplotlib versions and between machines. Presumably the pytest extension controls for this stuff by being prescriptive with the figure setup. I think we should give it a try - it could significantly lower the barrier to testing new plot features.

@rebeccafair rebeccafair added the nice to have Not urgently required, but would be an improvement label May 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
nice to have Not urgently required, but would be an improvement
Projects
None yet
Development

No branches or pull requests

2 participants