Skip to content

Fix prompt truncation with tool messages - #7655

Open
wmcgyver wants to merge 1 commit into
oobabooga:mainfrom
wmcgyver:fix-tool-fetch-crash
Open

Fix prompt truncation with tool messages#7655
wmcgyver wants to merge 1 commit into
oobabooga:mainfrom
wmcgyver:fix-tool-fetch-crash

Conversation

@wmcgyver

@wmcgyver wmcgyver commented Sep 1, 2026

Copy link
Copy Markdown

Summary

Fixes prompt truncation when tool messages follow the current user message.

When the prompt exceeds the context limit, the existing truncation logic can treat the current user message as old history and remove it. The following assistant tool-call and tool-result messages are then also removed, leaving messages empty and causing make_prompt() to fail at messages[-1].

This change tracks the most recent user message, only removes history preceding it, and truncates that user message directly when necessary.

Testing

  • python -m py_compile modules/chat.py
  • git diff --check
  • Tested with a ~239k-token input truncated to fit a 32k context.
  • Repeated successful web_search tool-call continuations after truncation.
  • Tested failed fetch_webpage calls followed by successful model continuation.
  • Tested with Qwen3.8-27B IQ3_M and IQ4_XS.
  • No empty messages / IndexError occurred after the change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant