Add chat history compaction feature with AI-based summarization

[robch]

Overview

This PR implements chat history compaction using AI-based summarization techniques to enable longer conversations within token limits.

Features

• New 'chat compact' command that intelligently summarizes older parts of conversation
• Multiple compaction modes (full, simple, none) with configurable behavior
• Ability to preserve recent messages intact while summarizing older history
• Configurable token targets for different components (chat, prompt, tool)
• Comprehensive help documentation for the new functionality

Implementation Details

• Added ChatCompactCommand to handle the command line interface
• Created ChatHistoryCompactionHelper for the core compaction logic
• Added configuration settings for default behavior
• Updated existing help files to reference the new capability

Usage

CYCOD - AI-powered Developer CLI, Version 1.0.0-DEV-r-20250909.0300+24133384b4920a9e84df3232942b956c7d8599d5
Copyright(c) 2025, Rob Chambers. All rights reserved.

Invalid argument: [options]

CYCOD CHAT COMPACT

Compact chat history files using AI-based summarization. This command
intelligently summarizes older parts of the conversation while preserving
important context, enabling much longer conversations within token limits.

USAGE: cycod chat compact [options]

ARGUMENTS

Chat history file to compact (required)
Can also be specified with --file

OPTIONS

--file Specify a chat history file to compact
--output Output file for the compacted history (default: original-name-compacted.jsonl)
--compact [mode] Specify compaction mode: full, simple, or none (default: full if --compact used without a mode)
--no-compact Disable compaction (same as --compact none)
--preserve-messages Number of recent messages to preserve intact (default: 4)
--preserve Same as --preserve-messages (for compatibility)
--max-tokens Maximum total chat tokens target (managed by CYCOD) (default: 160000)
--max-prompt-tokens Maximum prompt token target (managed by CYCOD) (default: 50000)
--max-tool-tokens Maximum tool token target (managed by CYCOD) (default: 50000)

COMPACTION MODES

For detailed information about compaction modes:
cycod help chat compaction modes

EXAMPLES

EXAMPLE 1: Compact a chat history file with default (full) mode

cycod chat compact chat-history.jsonl

EXAMPLE 2: Use simple mode and specify output file

cycod chat compact chat-history.jsonl --compact simple --output simple-chat.jsonl

EXAMPLE 3: Preserve more recent messages and set token target

cycod chat compact chat-history.jsonl --preserve-messages 6 --max-tokens 8000

EXAMPLE 4: Set different token limits for different token types

cycod chat compact chat-history.jsonl --max-prompt-tokens 16000 --max-tool-tokens 8000 --max-tokens 32000

EXAMPLE 5: Disable compaction

cycod chat compact chat-history.jsonl --no-compact

SEE ALSO

cycod help chat history
cycod help chat compaction modes
cycod help chat token management
cycod help settings
cycod help config

Related Issue

Closes #2509

[mercersoft]

Looks good. It follows the same 'cycod {noun} {verb} {options}' pattern as 'cycod config list'.

I did not study in detail the compation strategy in the helper. Happy to do it if you want me to though

[mercersoft]

I will create a new version for HOMEBREW so this becomes available to MacOS users.