fix: tool api call broken when user answer no to when asked for confirmation #371

jrmi · 2024-12-30T23:13:36Z

This MR fixes a few bug regarding tool call in tool format:

When the user choose answer on when asked for confirmation
When user edit the file before saving
When the user uses Ctrl+c to interrupt gptme

The underlying problems where that sometimes the call_id wasn't set in certain situations or multiple messages where returned by the tooluse.execute call but all were the response from the tool so we needed to merge them.

Important

Fixes tool API call issues by ensuring call_id is set and merging tool responses, with updates to message handling and tests.

Behavior:
- Fixes tool API call issues when users answer "no" to confirmation, edit files before saving, or interrupt with Ctrl+c.
- Ensures call_id is set correctly in execute_msg() in tools/__init__.py and ToolUse.execute() in tools/base.py.
- Merges multiple tool responses with the same call_id in _merge_tool_results_with_same_call_id() in llm_openai.py and llm_anthropic.py.
Functions:
- Updates _handle_tools() in llm_anthropic.py and llm_openai.py to handle user role messages with call_id.
- Modifies _transform_system_messages() in llm_anthropic.py to merge consecutive user messages with the same call_id.
Tests:
- Adds test cases in test_llm_anthropic.py and test_llm_openai.py to verify message conversion and tool handling with call_id.

^{This description was created by}^{for 2d25236. It will automatically update as commits are pushed.}

jrmi · 2024-12-30T23:17:18Z

gptme/chat.py

@@ -235,9 +235,6 @@ def step(
        if msg_response:
            yield msg_response.replace(quiet=True)
            yield from execute_msg(msg_response, confirm)
-    except KeyboardInterrupt:


This is now handled in the execute_msg as we need the call_id in the message response.

gptme/tools/base.py

ellipsis-dev

👍 Looks good to me! Reviewed everything up to 2d25236 in 1 minute and 12 seconds

More details

Looked at 223 lines of code in 7 files
Skipped 0 files when reviewing.
Skipped posting 2 drafted comments based on config settings.

1. gptme/llm/llm_anthropic.py:203

Draft comment:
The change from message["role"] == "system" to message["role"] == "user" seems incorrect. The original condition was likely correct as it was handling system messages with call_id. Please verify the intent of this change.
Reason this comment was not posted:
Comment did not seem useful.

2. gptme/llm/llm_openai.py:287

Draft comment:
The function _merge_tool_results_with_same_call_id is defined but not used in this file. Consider integrating it where necessary or removing it if not needed.
Reason this comment was not posted:
Comment looked like it was already resolved.

Workflow ID: wflow_Iq0Ct3a5VFKQWQwU

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

ErikBjare · 2025-01-04T16:06:35Z

gptme/chat.py

@@ -235,9 +235,6 @@ def step(
        if msg_response:
            yield msg_response.replace(quiet=True)
            yield from execute_msg(msg_response, confirm)
-    except KeyboardInterrupt:
-        clear_interruptible()
-        yield Message("system", "Interrupted")
    finally:
        clear_interruptible()


I'm not sure if this is correct, I think the clear_interruptible is no longer called?

I think the finally clause might not run for the generator until the next() call.

Nevermind, it's there in the except clause for that very reason, but no longer needed.

Not quite clear to me if the new behavior is correct though.

Hmm no it's probably not the same. Let me find something.

I've updated the MR. To stay on the safe side, I've called clear_interruptible in the execute_msg function. Notice the break in the except clause. Before it was still calling subsequent tools if any after a user interruption 🤦

We could also call clear_interruptible inside the handle_keyboard_interrupt itself just before raising the KeyboardInterrupt exception, what do you think?

ErikBjare · 2025-01-05T16:51:19Z

gptme/llm/llm_anthropic.py

+            messages_new
+            and messages_new[-1].role == "user"
+            and message.role == "user"
+            and message.call_id == messages_new[-1].call_id


Why is this condition needed?

When using the function call API, multiple calls may occur within a single answer (parallel calls). To handle this properly, ensure there is one tool response per tool call. By verifying that the call_id is distinct, we can ensure messages are not merged unless they originate from the same function call.

ErikBjare · 2025-01-05T16:52:52Z

gptme/tools/__init__.py

+                clear_interruptible()
+                yield Message(
+                    "system",
+                    "User hit Ctrl-c to interrupt the process",


I kinda want to make these shorter

Suggested change

"User hit Ctrl-c to interrupt the process",

"Interrupted by user",

Sort of what I did in #384

I changed the message (and used a constant to have the same value everywhere it's needed.)

jrmi changed the title ~~Fix tool api call broken when no tool answer~~ fix: tool api call broken when user answer no to confirmation Dec 30, 2024

jrmi commented Dec 30, 2024

View reviewed changes

gptme/tools/base.py Show resolved Hide resolved

jrmi marked this pull request as ready for review December 31, 2024 10:05

ellipsis-dev bot reviewed Dec 31, 2024

View reviewed changes

jrmi changed the title ~~fix: tool api call broken when user answer no to confirmation~~ fix: tool api call broken when user answer no to when asked for confirmation Dec 31, 2024

jrmi force-pushed the fix-tool-api-call-broken-when-no-tool-answer branch from 2d25236 to ac85af3 Compare January 4, 2025 15:18

ErikBjare reviewed Jan 4, 2025

View reviewed changes

jrmi force-pushed the fix-tool-api-call-broken-when-no-tool-answer branch 2 times, most recently from f09caaf to 871c76e Compare January 4, 2025 16:39

ErikBjare reviewed Jan 5, 2025

View reviewed changes

ErikBjare mentioned this pull request Jan 5, 2025

fix: fixed behavior for keyboard interrupts and user commands #384

Open

jrmi added 3 commits January 5, 2025 18:51

fix: tool response broken after user interruption

4873bb6

fix: broken tool call after editing the file before saving

8be2b4f

fix: shorten interrupt message content

420b2d9

jrmi force-pushed the fix-tool-api-call-broken-when-no-tool-answer branch from 871c76e to 420b2d9 Compare January 5, 2025 18:08

jrmi requested a review from ErikBjare January 5, 2025 18:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: tool api call broken when user answer no to when asked for confirmation #371

fix: tool api call broken when user answer no to when asked for confirmation #371

jrmi commented Dec 30, 2024 •

edited by ellipsis-dev bot

Loading

jrmi Dec 30, 2024

ellipsis-dev bot left a comment

ErikBjare Jan 4, 2025

ErikBjare Jan 4, 2025 •

edited

Loading

jrmi Jan 4, 2025

jrmi Jan 4, 2025

jrmi Jan 4, 2025

ErikBjare Jan 5, 2025

jrmi Jan 5, 2025 •

edited

Loading

ErikBjare Jan 5, 2025

ErikBjare Jan 5, 2025

jrmi Jan 5, 2025

	"User hit Ctrl-c to interrupt the process",
	"Interrupted by user",

fix: tool api call broken when user answer no to when asked for confirmation #371

Are you sure you want to change the base?

fix: tool api call broken when user answer no to when asked for confirmation #371

Conversation

jrmi commented Dec 30, 2024 • edited by ellipsis-dev bot Loading

Choose a reason for hiding this comment

ellipsis-dev bot left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ErikBjare Jan 4, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jrmi Jan 5, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jrmi commented Dec 30, 2024 •

edited by ellipsis-dev bot

Loading

ErikBjare Jan 4, 2025 •

edited

Loading

jrmi Jan 5, 2025 •

edited

Loading