Skip to content

Commit bb9884d

Browse files
authored
Print maximum relative error when checksums fail (#6252)
This PR adds a print of the maximum relative error when checksums fail, which I think is quite useful when they fail for many "keys" and/or many tests (e.g., #5955, #6248) - the maximum relative error gives a glimpse of how good or bad things are. This is an example of the new output, with the extra line printing the maximum relative error: ``` ... ERROR: Benchmark and output file checksum have different value for key [lev=0,Ex] Benchmark: [lev=0,Ex] 9.408402900367834e+05 Test file: [lev=0,Ex] 9.409402900367834e+05 Absolute error: 1.00e+02 Relative error: 1.06e-04 ERROR: Benchmark and output file checksum have different value for key [lev=0,Ez] Benchmark: [lev=0,Ez] 6.654423300420902e+05 Test file: [lev=0,Ez] 6.754423300420902e+05 Absolute error: 1.00e+04 Relative error: 1.50e-02 ERROR: Benchmark and output file checksum have different value for key [lev=0,jx] Benchmark: [lev=0,jx] 9.956269465957083e+03 Test file: [lev=0,jx] 9.956269365957083e+03 Absolute error: 1.00e-04 Relative error: 1.00e-08 Maximum relative error: 1.50e-02 New checksums file test_2d_larmor.json: ... ``` Feel free to close the PR without merging if you think this is not useful.
1 parent 4a62fe0 commit bb9884d

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

Regression/Checksum/checksum.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -265,6 +265,7 @@ def evaluate(self, rtol=1.0e-9, atol=1.0e-40):
265265

266266
# Dictionaries have same values?
267267
checksums_differ = False
268+
max_rel_err = 0.0
268269
for key1 in ref_benchmark.data.keys():
269270
for key2 in ref_benchmark.data[key1].keys():
270271
passed = np.isclose(
@@ -294,6 +295,8 @@ def evaluate(self, rtol=1.0e-9, atol=1.0e-40):
294295
if np.abs(x) != 0.0:
295296
rel_err = abs_err / np.abs(x)
296297
print("Relative error: {:.2e}".format(rel_err))
298+
max_rel_err = max(max_rel_err, rel_err)
299+
print("\nMaximum relative error: {:.2e}".format(max_rel_err))
297300
if checksums_differ:
298301
print(f"\nNew checksums file {self.test_name}.json:")
299302
print(json.dumps(self.data, indent=2))

0 commit comments

Comments
 (0)