Problem Description
When Auto Compaction is triggered, it compacts the entire message history. This can make it difficult for the LLM to transition from the compaction to the current prompt.
Proposed Solution
If Auto Compaction instead left some of the most recent messages intact, they could serve as a bridge which could help the LLM more smoothly transition from the compaction to the current prompt. I propose adding a setting under Chat Settings > Context Management to allow the user to set how many of the most recent messages are preserved and not compacted.
Additional Context
For example, if the user sets Chatbox to preserve the 5 most recent messages, and has a conversation that is 100 messages long when Auto Compaction triggers, the 95 oldest messages would be compacted. The user's next prompt would then send the compaction of the old messages, the 5 recent messages, and the current prompt. The 5 uncompacted messages would give the LLM needed details to smoothly bridge the compaction with the continued conversation.
Problem Description
When Auto Compaction is triggered, it compacts the entire message history. This can make it difficult for the LLM to transition from the compaction to the current prompt.
Proposed Solution
If Auto Compaction instead left some of the most recent messages intact, they could serve as a bridge which could help the LLM more smoothly transition from the compaction to the current prompt. I propose adding a setting under Chat Settings > Context Management to allow the user to set how many of the most recent messages are preserved and not compacted.
Additional Context
For example, if the user sets Chatbox to preserve the 5 most recent messages, and has a conversation that is 100 messages long when Auto Compaction triggers, the 95 oldest messages would be compacted. The user's next prompt would then send the compaction of the old messages, the 5 recent messages, and the current prompt. The 5 uncompacted messages would give the LLM needed details to smoothly bridge the compaction with the continued conversation.