Fix/tool call accuracy (#2079)

[sdivye92]

• Fix ToolCallAccuracy returns score higher than 1.0 (ToolCallAccuracy returns score higher than 1.0 #2079)
  • Using zip to combine reference_tool_calls and pred_tool_calls
• Updated is_sequence_aligned method to check for same len of pred and ref seq
  • This fixes issue when tool call accuracy is 1 even when pred_tool_calls has more tool calls than reference_tool_calls.
  • equal check between pred_sequence and ref_sequence will check that both should be same length.
  • when equal order of occurence of their elements are same.

[greptile-apps]

PR Summary

Fixed critical scoring bug in ToolCallAccuracy metric where it could incorrectly return scores higher than 1.0 due to double-counting matches.

• Modified ragas/src/ragas/metrics/_tool_call_accuracy.py to use zip() for one-to-one matching between reference and prediction tool calls
• Updated is_sequence_aligned to enforce equal lengths between prediction and reference sequences
• Added validation to ensure tool calls are made in correct order and quantity
• Fixed edge case where perfect scores were given despite extra predicted tool calls

_{1 file reviewed, 1 comment
Edit PR Review Bot Settings | Greptile}