Golden File Testing

[quimpm]

Would be nice to make JUnit able to support Golden File Testing.

Golden File Testing is an excellent approach for testing huge output files, API responses... It's useful when, apart from testing all the logic of your program (this is done by unit testing itself), furthermore you also want to know that what you are outputting and giving to a client, downstream, whatever... it's fine and contains what it should. Maybe in that impasse of writing the data into a file, something is not working properly.

I thought that maybe it can be done by adding an optional when calling to JUnit like --accept in order to write the result of the test to the golden file and, when that option isn't set, execute normal golden test behavior.

I thought also about different solutions like doing a library apart from JUnit and then use the approach inside unit tests. But there's always the dependency with JUnit due to the acceptance parameter when we want the test to write into the golden file. For example, if this done through an annotation in each TestCase class, every time you want to accept the output and write it to the golden files, you will have to change each TestClass... And that is a bit painful if you have a huge project with a lot of test cases.

Some links about Golden File Testing:
https://ro-che.info/articles/2017-12-04-golden-tests
https://medium.com/@jarifibrahim/golden-files-why-you-should-use-them-47087ec994bf
https://ieftimov.com/post/testing-in-go-golden-files/

[smoyer64]

Golden files are indeed useful and broadly used in Go testing. There's a great presentation by Mitchell Hashimoto (of Hashicorp) for the uninitiated and I agree that it would be useful in Java as well. If it's not part of JUnit, perhaps it can find a place in JUnitPioneer as a community supported extension.

cc: @nipafx