|
| 1 | +# Fix: OpenAI API Error "messages with role 'tool' must be a response to a preceding message with 'tool_calls'" |
| 2 | + |
| 3 | +## Problem Description |
| 4 | +In the `async-context-compression` filter, chat history can be trimmed or summarized when the conversation grows. If the retained tail starts in the middle of a native tool-calling sequence, the next request may begin with a `tool` message whose triggering `assistant` message is no longer present. |
| 5 | + |
| 6 | +That produces the OpenAI API error: |
| 7 | +`"messages with role 'tool' must be a response to a preceding message with 'tool_calls'"` |
| 8 | + |
| 9 | +## Root Cause |
| 10 | +History compression boundaries were not fully aware of atomic tool-call chains. A valid chain may include: |
| 11 | + |
| 12 | +1. An `assistant` message with `tool_calls` |
| 13 | +2. One or more `tool` messages |
| 14 | +3. An optional assistant follow-up that consumes the tool results |
| 15 | + |
| 16 | +If truncation happens inside that chain, the request sent to the model becomes invalid. |
| 17 | + |
| 18 | +## Solution: Atomic Boundary Alignment |
| 19 | +The fix groups tool-call sequences into atomic units and aligns trim boundaries to those groups. |
| 20 | + |
| 21 | +### 1. `_get_atomic_groups()` |
| 22 | +This helper groups message indices into units that must be kept or dropped together. It explicitly recognizes native tool-calling patterns such as: |
| 23 | + |
| 24 | +- `assistant(tool_calls)` |
| 25 | +- `tool` |
| 26 | +- assistant follow-up response |
| 27 | + |
| 28 | +Conceptually, it treats the whole sequence as one atomic block instead of independent messages. |
| 29 | + |
| 30 | +```python |
| 31 | +def _get_atomic_groups(self, messages: List[Dict]) -> List[List[int]]: |
| 32 | + groups = [] |
| 33 | + current_group = [] |
| 34 | + |
| 35 | + for i, msg in enumerate(messages): |
| 36 | + role = msg.get("role") |
| 37 | + has_tool_calls = bool(msg.get("tool_calls")) |
| 38 | + |
| 39 | + if role == "assistant" and has_tool_calls: |
| 40 | + if current_group: |
| 41 | + groups.append(current_group) |
| 42 | + current_group = [i] |
| 43 | + elif role == "tool": |
| 44 | + if not current_group: |
| 45 | + groups.append([i]) |
| 46 | + else: |
| 47 | + current_group.append(i) |
| 48 | + elif ( |
| 49 | + role == "assistant" |
| 50 | + and current_group |
| 51 | + and messages[current_group[-1]].get("role") == "tool" |
| 52 | + ): |
| 53 | + current_group.append(i) |
| 54 | + groups.append(current_group) |
| 55 | + current_group = [] |
| 56 | + else: |
| 57 | + if current_group: |
| 58 | + groups.append(current_group) |
| 59 | + current_group = [] |
| 60 | + groups.append([i]) |
| 61 | + |
| 62 | + if current_group: |
| 63 | + groups.append(current_group) |
| 64 | + |
| 65 | + return groups |
| 66 | +``` |
| 67 | + |
| 68 | +### 2. `_align_tail_start_to_atomic_boundary()` |
| 69 | +This helper checks whether a proposed trim point falls inside one of those atomic groups. If it does, the start index is moved backward to the beginning of that group. |
| 70 | + |
| 71 | +```python |
| 72 | +def _align_tail_start_to_atomic_boundary( |
| 73 | + self, messages: List[Dict], raw_start_index: int, protected_prefix: int |
| 74 | +) -> int: |
| 75 | + aligned_start = max(raw_start_index, protected_prefix) |
| 76 | + |
| 77 | + if aligned_start <= protected_prefix or aligned_start >= len(messages): |
| 78 | + return aligned_start |
| 79 | + |
| 80 | + trimmable = messages[protected_prefix:] |
| 81 | + local_start = aligned_start - protected_prefix |
| 82 | + |
| 83 | + for group in self._get_atomic_groups(trimmable): |
| 84 | + group_start = group[0] |
| 85 | + group_end = group[-1] + 1 |
| 86 | + |
| 87 | + if local_start == group_start: |
| 88 | + return aligned_start |
| 89 | + |
| 90 | + if group_start < local_start < group_end: |
| 91 | + return protected_prefix + group_start |
| 92 | + |
| 93 | + return aligned_start |
| 94 | +``` |
| 95 | + |
| 96 | +### 3. Applied to Tail Retention and Summary Progress |
| 97 | +The aligned boundary is now used when rebuilding the retained tail and when calculating how much history can be summarized safely. |
| 98 | + |
| 99 | +Example from the current implementation: |
| 100 | + |
| 101 | +```python |
| 102 | +raw_start_index = max(compressed_count, effective_keep_first) |
| 103 | +start_index = self._align_tail_start_to_atomic_boundary( |
| 104 | + messages, raw_start_index, effective_keep_first |
| 105 | +) |
| 106 | +tail_messages = messages[start_index:] |
| 107 | +``` |
| 108 | + |
| 109 | +And during summary progress calculation: |
| 110 | + |
| 111 | +```python |
| 112 | +raw_target_compressed_count = max(0, len(messages) - self.valves.keep_last) |
| 113 | +target_compressed_count = self._align_tail_start_to_atomic_boundary( |
| 114 | + messages, raw_target_compressed_count, effective_keep_first |
| 115 | +) |
| 116 | +``` |
| 117 | + |
| 118 | +## Verification Results |
| 119 | +- **First compression boundary**: When history first crosses the compression threshold, the retained tail no longer starts inside a tool-call block. |
| 120 | +- **Complex sessions**: Real-world testing with 30+ messages, multiple tool calls, and failed calls remained stable during background summarization. |
| 121 | +- **Regression behavior**: The filter now prefers a valid boundary even if that means retaining slightly more context than a naive raw slice would allow. |
| 122 | + |
| 123 | +## Conclusion |
| 124 | +The fix prevents orphaned `tool` messages by making history trimming and summary progress aware of atomic tool-call groups. This eliminates the 400 error during long conversations and background compression. |
0 commit comments