Report tool-model / test-parse failures in test-case validation; fix Infinity in_range bound#23004
Conversation
1c5e695 to
ef115c8
Compare
…Infinity bound validate_test_cases_for_tool_source() builds the parameter model and parses the test cases before its per-test loop, outside any try/except. A tool whose model cannot be built (e.g. an unknown parameter type) or whose <test> block is malformed therefore raised an unhandled exception out of the whole call, so a caller validating tools it does not control (the 24.2 upgrade advisor, planemo / lint, CI) aborted instead of reporting the tool. Wrap the model build + test parsing so a failure becomes a single reported validation result. This addresses the model-construction / test-parsing layer (31 tools across the public Galaxy tool corpus); together with the per-test state-building fix it takes the validator from 159 crashing tools to 15, and the remaining 15 fail earlier in get_tool_source (malformed XML, or a macro file missing from the checkout) before this function is reached. Also fix _parse_number: an in_range validator bound written 'Infinity' (or 'Inf', or scientific notation like '1E5') fell through the case-sensitive float check to int(), which raised, even though float() parses it. Match those forms case-insensitively so such a tool validates instead of crashing. Add regression tests for an unmodelable tool and an Infinity in_range bound.
|
CI has settled with five failing jobs. All are in subsystems this PR does not touch — the change is confined to
None exercise tool test-case validation or validator-bound parsing. Happy to rebase once the |
| try: | ||
| tool_parameter_bundle = input_models_for_tool_source(tool_source) | ||
| test_cases: List[ToolSourceTest] = tool_source.parse_tests_to_dict()["tests"] | ||
| except Exception as e: |
There was a problem hiding this comment.
This is minor, but could you narrow this exception to ParameterDefinitionError and ValueError?
guerler
left a comment
There was a problem hiding this comment.
Looks good to me! Please rebase to resolve conflicts.
Fixes #23003.
validate_test_cases_for_tool_source()builds the parameter model (input_models_for_tool_source) and parses the test cases (parse_tests_to_dict) before its per-test loop, outside anytry/except. A tool whose model cannot be built (e.g. an unknown parameter type) or whose<test>block is malformed therefore raised an unhandled exception out of the whole call, so a caller validating tools it does not control (the 24.2 upgrade advisor, planemo / lint, CI) aborted instead of reporting the tool.This change has two parts:
Report, don't raise (holistic). Wrap the model build + test parsing so a failure becomes a single reported
TestCaseStateValidationResultwith the error, rather than escaping. This addresses the model-construction / test-parsing layer — 31 tools across the public Galaxy tool corpus._parse_numberInfinityfix (targeted). Anin_rangevalidator bound writtenInfinity(orInf, or scientific notation like1E5) fell through the case-sensitive float check ("inf" in raw_value) toint(), which raised, even thoughfloat()parses it. The check is now case-insensitive, so such a tool validates instead of crashing.This complements #23002, which fixes the same "report, don't raise" gap one layer in (per-test state building). With both, the validator goes from 159 crashing corpus tools to 15; the remaining 15 fail earlier in
get_tool_source(malformed XML, or a macro file missing from the checkout), before this function is reached.How to test the changes?
test/unit/tool_util/test_parameter_test_cases.py:test_unmodelable_tool_is_reported_not_raised— a tool with an unknown parameter type is reported (validation_error), not raised.test_infinity_in_range_validator_bound_does_not_crash— a tool with anin_rangemax="Infinity"validates cleanly.License