Feature: Intelligent Tool Management System (Multi-Provider)

## Summary

Implement a comprehensive **Intelligent Tool Management System** that handles 100+ tools efficiently across multiple AI providers (Anthropic, OpenAI, Gemini). This includes on-demand tool discovery, provider-specific optimizations, policy-based filtering, and result truncation.

**Consolidates**: #69 (Tool Profiles), #70 (Budget Management), #67 (Result Truncation)

---

## Problem

When many tools are registered:
- **Context bloat**: 80 tools × ~500 tokens = 40,000+ tokens overhead
- **Selection accuracy degrades** after 30-50 tools
- **Provider differences**: Each AI provider has different tool calling quirks
- **No unified management**: Tools scattered without organization
- **Result overhead**: Large tool results consume context

### Real-World Impact
| Scenario | Token Overhead |
|----------|---------------|
| 5 MCP servers | ~55K tokens |
| 80 custom tools | ~40K tokens |
| Large results | +10-50K tokens |

---

## Solution Overview

```
┌─────────────────────────────────────────────────────────────────────┐
│                    Tool Management System                           │
├─────────────────────────────────────────────────────────────────────┤
│                                                                     │
│  ┌──────────────┐   ┌──────────────┐   ┌──────────────────────────┐│
│  │ Active Tools │   │   Deferred   │   │    Tool Profiles         ││
│  │  (Always On) │   │   (Search)   │   │  coding, minimal, full   ││
│  └──────────────┘   └──────────────┘   └──────────────────────────┘│
│         │                  │                      │                 │
│         └──────────────────┴──────────────────────┘                 │
│                            │                                        │
│                    ┌───────▼───────┐                                │
│                    │ Policy Filter │                                │
│                    │ (allow/deny)  │                                │
│                    └───────┬───────┘                                │
│                            │                                        │
│         ┌──────────────────┼──────────────────┐                     │
│         ▼                  ▼                  ▼                     │
│  ┌────────────┐    ┌────────────┐    ┌────────────┐                │
│  │  Anthropic │    │   OpenAI   │    │   Gemini   │                │
│  │ formatter  │    │ formatter  │    │ formatter  │                │
│  └────────────┘    └────────────┘    └────────────┘                │
│                                                                     │
│  ┌──────────────────────────────────────────────────────────────┐  │
│  │              Result Truncation & Budget                       │  │
│  └──────────────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────────────┘
```

---

## 1. Tool Search & Deferred Loading

### 1.1 ToolDefinition Extensions

```typescript
interface ToolDefinition<TParams = Record<string, unknown>> {
  name: string;
  description: string;
  // ... existing fields ...
  
  // === NEW: Deferred Loading ===
  /** Defer loading - discovered via tool_search */
  deferLoading?: boolean;
  
  // === NEW: Organization ===
  /** Category for search/filtering */
  category?: string;
  /** Profile memberships */
  profiles?: string[];
  /** Search keywords */
  searchKeywords?: string[];
  
  // === NEW: Result Control ===
  /** Max result tokens (truncate if exceeded) */
  maxResultTokens?: number;
  /** Custom truncation strategy */
  truncationStrategy?: "head" | "tail" | "smart";
}
```

### 1.2 Tool Search Implementation

```typescript
// BM25 keyword search (works with all providers)
function searchTools(
  tools: ToolDefinition[],
  query: string,
  config?: { maxResults?: number; minScore?: number }
): ToolSearchResult[] {
  const { maxResults = 5, minScore = 0.1 } = config ?? {};
  const queryTerms = tokenize(query);
  const idf = calculateIDF(tools, queryTerms);
  
  return tools
    .map(tool => ({
      ...tool,
      score: calculateBM25Score(tool, queryTerms, idf)
    }))
    .filter(t => t.score >= minScore)
    .sort((a, b) => b.score - a.score)
    .slice(0, maxResults);
}

// Regex search (Anthropic/Vercel pattern)
function searchToolsByRegex(
  tools: ToolDefinition[],
  pattern: string
): ToolDefinition[] {
  const regex = new RegExp(pattern, "i");
  return tools.filter(t => 
    regex.test(t.name) || 
    regex.test(t.description) ||
    regex.test(t.category ?? "")
  );
}
```

### 1.3 tool_search Meta-Tool

```typescript
const toolSearchTool: ToolDefinition = {
  name: "tool_search",
  description: `Search for tools by keyword. Use when you need a capability not currently available.
Returns up to 5 relevant tools that become available immediately.

Examples:
- "upload training files" → file tools
- "FAQ knowledge" → training tools
- "conversation analytics" → analytics tools`,
  inputSchema: {
    type: "object",
    properties: {
      query: { type: "string", description: "Search keywords (2-5 words)" },
      method: { type: "string", enum: ["keyword", "regex"], default: "keyword" }
    },
    required: ["query"]
  },
  handler: async ({ query, method }) => {
    const results = method === "regex" 
      ? searchToolsByRegex(deferredTools, query)
      : searchTools(deferredTools, query);
    
    // Load discovered tools
    loadDeferredTools(results);
    
    return {
      success: true,
      message: `Found ${results.length} tools. Now available for use.`,
      data: { tools: results.map(t => ({ name: t.name, description: t.description })) }
    };
  }
};
```

---

## 2. Multi-Provider Support

### 2.1 Provider Comparison

| Feature | Anthropic | OpenAI | Gemini |
|---------|-----------|--------|--------|
| Native defer_loading | ✅ Yes | ✅ GPT-5.4+ | ❌ No |
| Tool filtering | Via tool_search | `allowed_tools` | Schema filtering |
| Strict mode | ✅ | ✅ (recommended) | ✅ |
| Max tools | No limit | ~100 optimal | No limit |
| Schema quirks | Nested objects OK | No root unions | Strip constraints |

### 2.2 Provider Formatters

```typescript
class ToolManager {
  formatForProvider(provider: AIProvider): FormattedTools {
    const active = this.getActiveTools();
    
    switch (provider) {
      case "anthropic":
        return this.formatForAnthropic(active);
      case "openai":
        return this.formatForOpenAI(active);
      case "gemini":
        return this.formatForGemini(active);
      default:
        return { tools: active };
    }
  }
  
  private formatForAnthropic(tools: ToolDefinition[]) {
    // Anthropic: Add tool_search, use defer_loading natively
    const hasDeferred = this.deferredTools.size > 0;
    
    return {
      tools: tools.map(t => ({
        name: t.name,
        description: t.description,
        input_schema: t.inputSchema,
        // Native defer_loading support
        ...(t.deferLoading && { cache_control: { type: "ephemeral" } })
      })),
      ...(hasDeferred && { 
        tool_search: this.createToolSearchTool() 
      })
    };
  }
  
  private formatForOpenAI(tools: ToolDefinition[]) {
    // OpenAI: Use allowed_tools for filtering, strict mode
    return {
      tools: tools.map(t => ({
        type: "function",
        function: {
          name: t.name,
          description: t.description,
          parameters: t.inputSchema,
          strict: true // Recommended
        }
      })),
      tool_choice: { type: "auto" }
    };
  }
  
  private formatForGemini(tools: ToolDefinition[]) {
    // Gemini: Strip constraint keywords, clean schemas
    return {
      tools: [{
        functionDeclarations: tools.map(t => ({
          name: t.name,
          description: t.description,
          parameters: this.cleanSchemaForGemini(t.inputSchema)
        }))
      }]
    };
  }
  
  private cleanSchemaForGemini(schema: ToolInputSchema): object {
    // Remove unsupported keywords
    const cleaned = { ...schema };
    delete cleaned.minLength;
    delete cleaned.maxLength;
    delete cleaned.pattern;
    delete cleaned.minimum;
    delete cleaned.maximum;
    return cleaned;
  }
}
```

---

## 3. Tool Profiles & Policies

### 3.1 Profile System (from #69)

```typescript
interface ToolProfile {
  name: string;
  description?: string;
  include?: string[];  // Tool names or "category:*" patterns
  exclude?: string[];
}

const BUILT_IN_PROFILES: Record<string, ToolProfile> = {
  minimal: {
    name: "minimal",
    include: ["navigate", "search", "read"],
  },
  coding: {
    name: "coding",
    include: ["category:filesystem", "category:code", "search"],
    exclude: ["admin_*"],
  },
  training: {
    name: "training",
    include: ["category:training", "category:faq", "navigate"],
  },
  full: {
    name: "full",
    // Include everything
  }
};

// Usage
chat.setToolProfile("coding");
```

### 3.2 Policy Pipeline (OpenClaw Pattern)

```typescript
interface PolicyStep {
  policy: ToolPolicy;
  label: string;
}

function applyToolPolicyPipeline(params: {
  tools: ToolDefinition[];
  steps: PolicyStep[];
}): ToolDefinition[] {
  let filtered = params.tools;
  
  for (const step of params.steps) {
    if (!step.policy) continue;
    
    filtered = filtered.filter(tool => {
      // Check allow list
      if (step.policy.allow && !step.policy.allow.includes(tool.name)) {
        return false;
      }
      // Check deny list
      if (step.policy.deny?.includes(tool.name)) {
        return false;
      }
      // Check category patterns
      if (step.policy.allowCategories) {
        return step.policy.allowCategories.includes(tool.category);
      }
      return true;
    });
  }
  
  return filtered;
}

// Usage (layered policies)
const tools = applyToolPolicyPipeline({
  tools: allTools,
  steps: [
    { policy: profilePolicy, label: "profile" },
    { policy: globalPolicy, label: "global" },
    { policy: agentPolicy, label: "agent" },
    { policy: userPolicy, label: "user" },
  ]
});
```

---

## 4. Result Truncation & Budget (from #67, #70)

### 4.1 Token Budget Management

```typescript
interface TokenBudget {
  /** Max tokens for tool definitions */
  toolDefinitions?: number;  // Default: 10000
  /** Max tokens per tool result */
  perToolResult?: number;    // Default: 2000
  /** Max total tokens for all results in loop */
  totalResults?: number;     // Default: 8000
}

class BudgetManager {
  private budget: TokenBudget;
  private usedTokens = { definitions: 0, results: 0 };
  
  shouldDeferTool(tool: ToolDefinition): boolean {
    const toolTokens = estimateToolTokens(tool);
    return this.usedTokens.definitions + toolTokens > this.budget.toolDefinitions!;
  }
  
  truncateResult(result: ToolResponse, tool: ToolDefinition): ToolResponse {
    const maxTokens = tool.maxResultTokens ?? this.budget.perToolResult!;
    const resultTokens = estimateTokens(JSON.stringify(result));
    
    if (resultTokens <= maxTokens) return result;
    
    return this.applyTruncation(result, maxTokens, tool.truncationStrategy);
  }
  
  private applyTruncation(
    result: ToolResponse, 
    maxTokens: number,
    strategy: "head" | "tail" | "smart" = "smart"
  ): ToolResponse {
    if (strategy === "smart") {
      // Keep structure, truncate large arrays/strings
      return this.smartTruncate(result, maxTokens);
    }
    // Simple head/tail truncation
    const json = JSON.stringify(result.data);
    const truncated = strategy === "head" 
      ? json.slice(0, maxTokens * 4)
      : json.slice(-maxTokens * 4);
    
    return {
      ...result,
      data: JSON.parse(truncated),
      _truncated: true
    };
  }
  
  private smartTruncate(result: ToolResponse, maxTokens: number): ToolResponse {
    // Intelligent truncation that preserves structure
    // - Keep first/last N items of arrays
    // - Summarize long strings
    // - Add "[...N more items]" indicators
    // Implementation details...
  }
}
```

### 4.2 Auto-Summarization for Large Results

```typescript
interface SummarizationConfig {
  /** Threshold to trigger summarization */
  tokenThreshold: number;
  /** Summary prompt */
  summaryPrompt?: string;
}

async function summarizeToolResult(
  result: ToolResponse,
  config: SummarizationConfig
): Promise<ToolResponse> {
  const tokens = estimateTokens(JSON.stringify(result));
  
  if (tokens < config.tokenThreshold) return result;
  
  // Use smaller model for summarization
  const summary = await generateSummary(result, config.summaryPrompt);
  
  return {
    success: true,
    message: result.message,
    data: {
      _summarized: true,
      _originalTokens: tokens,
      summary
    }
  };
}
```

---

## 5. Implementation

### Files to Create

| File | Purpose |
|------|---------|
| `src/core/tools/toolManager.ts` | Main ToolManager class |
| `src/core/tools/toolSearch/index.ts` | Search functions |
| `src/core/tools/toolSearch/bm25.ts` | BM25 algorithm |
| `src/core/tools/toolProfiles.ts` | Profile management |
| `src/core/tools/toolPolicy.ts` | Policy pipeline |
| `src/core/tools/providers/anthropic.ts` | Anthropic formatter |
| `src/core/tools/providers/openai.ts` | OpenAI formatter |
| `src/core/tools/providers/gemini.ts` | Gemini formatter |
| `src/core/tools/budget.ts` | Token budget management |
| `src/core/tools/truncation.ts` | Result truncation |
| `src/react/hooks/useToolSearch.ts` | React hook |
| `src/react/hooks/useToolProfile.ts` | Profile hook |

### Files to Modify

| File | Changes |
|------|---------|
| `src/core/types/tools.ts` | Add new fields |
| `src/chat/ChatWithTools.ts` | Integrate ToolManager |
| `src/react/provider/CopilotProvider.tsx` | Add context methods |

---

## 6. API Examples

### Basic Usage

```typescript
// Register tools with deferred loading
useTool({
  name: "add_faq",
  description: "Add FAQ to training",
  deferLoading: true,        // Won't be sent initially
  category: "training",      // For search/profiles
  profiles: ["training"],    // Profile membership
  maxResultTokens: 500,      // Truncate large results
  // ...
});

// Enable tool search
useToolSearch({ enabled: true, maxResults: 5 });

// Or set a profile
useToolProfile("coding");
```

### Advanced Usage

```typescript
// Custom policy
const chat = createChatWithTools({
  runtimeUrl: "/api/chat",
  toolManager: {
    profiles: {
      custom: {
        include: ["search", "category:training"],
        exclude: ["admin_*"]
      }
    },
    policies: [
      { allow: ["read", "write"], label: "base" },
      { deny: ["delete_*"], label: "safety" }
    ],
    budget: {
      toolDefinitions: 8000,
      perToolResult: 1500,
      totalResults: 6000
    },
    providers: {
      anthropic: { deferLoading: true },
      openai: { strictMode: true }
    }
  }
});
```

---

## 7. Token Savings

| Scenario | Before | After | Savings |
|----------|--------|-------|---------|
| 80 tools | ~40K | ~3K | **92%** |
| Large results (10 calls) | ~50K | ~15K | **70%** |
| Combined | ~90K | ~18K | **80%** |

---

## 8. Checklist

### Phase 1: Core Infrastructure
- [ ] Extend ToolDefinition with new fields
- [ ] Implement ToolManager class
- [ ] BM25 search algorithm
- [ ] Regex search
- [ ] tool_search meta-tool
- [ ] Token estimation utilities

### Phase 2: Multi-Provider
- [ ] Anthropic formatter (defer_loading)
- [ ] OpenAI formatter (allowed_tools, strict)
- [ ] Gemini formatter (schema cleaning)
- [ ] Provider detection & auto-formatting

### Phase 3: Profiles & Policies
- [ ] Profile system
- [ ] Built-in profiles (minimal, coding, full)
- [ ] Policy pipeline
- [ ] Category-based filtering

### Phase 4: Budget & Truncation
- [ ] Token budget tracking
- [ ] Per-tool result limits
- [ ] Smart truncation
- [ ] Auto-summarization (optional)

### Phase 5: React Integration
- [ ] useToolSearch hook
- [ ] useToolProfile hook
- [ ] useToolBudget hook
- [ ] DevTools integration

---

## References

- [Anthropic: Advanced Tool Use](https://www.anthropic.com/engineering/advanced-tool-use)
- [Anthropic: Tool Search Docs](https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool)
- [OpenAI: Function Calling](https://platform.openai.com/docs/guides/function-calling)
- [Google: Gemini Tools](https://ai.google.dev/gemini-api/docs/tools)
- [Vercel AI SDK 6](https://vercel.com/blog/ai-sdk-6)
- [OpenClaw Tool System](https://github.com/open-claw/open-claw)
- [BM25 Algorithm](https://en.wikipedia.org/wiki/Okapi_BM25)

Scenario	Token Overhead
5 MCP servers	~55K tokens
80 custom tools	~40K tokens
Large results	+10-50K tokens

Feature	Anthropic	OpenAI	Gemini
Native defer_loading	✅ Yes	✅ GPT-5.4+	❌ No
Tool filtering	Via tool_search	`allowed_tools`	Schema filtering
Strict mode	✅	✅ (recommended)	✅
Max tools	No limit	~100 optimal	No limit
Schema quirks	Nested objects OK	No root unions	Strip constraints

File	Purpose
`src/core/tools/toolManager.ts`	Main ToolManager class
`src/core/tools/toolSearch/index.ts`	Search functions
`src/core/tools/toolSearch/bm25.ts`	BM25 algorithm
`src/core/tools/toolProfiles.ts`	Profile management
`src/core/tools/toolPolicy.ts`	Policy pipeline
`src/core/tools/providers/anthropic.ts`	Anthropic formatter
`src/core/tools/providers/openai.ts`	OpenAI formatter
`src/core/tools/providers/gemini.ts`	Gemini formatter
`src/core/tools/budget.ts`	Token budget management
`src/core/tools/truncation.ts`	Result truncation
`src/react/hooks/useToolSearch.ts`	React hook
`src/react/hooks/useToolProfile.ts`	Profile hook

File	Changes
`src/core/types/tools.ts`	Add new fields
`src/chat/ChatWithTools.ts`	Integrate ToolManager
`src/react/provider/CopilotProvider.tsx`	Add context methods

Scenario	Before	After	Savings
80 tools	~40K	~3K	92%
Large results (10 calls)	~50K	~15K	70%
Combined	~90K	~18K	80%

Feature: Intelligent Tool Management System (Multi-Provider) #72

Description

Summary

Problem

Real-World Impact

Solution Overview

1. Tool Search & Deferred Loading

1.1 ToolDefinition Extensions

1.2 Tool Search Implementation

1.3 tool_search Meta-Tool

2. Multi-Provider Support

2.1 Provider Comparison

2.2 Provider Formatters

3. Tool Profiles & Policies

3.1 Profile System (from #69)

3.2 Policy Pipeline (OpenClaw Pattern)

4. Result Truncation & Budget (from #67, #70)

4.1 Token Budget Management

4.2 Auto-Summarization for Large Results

5. Implementation

Files to Create

Files to Modify

6. API Examples

Basic Usage

Advanced Usage

7. Token Savings

8. Checklist

Phase 1: Core Infrastructure

Phase 2: Multi-Provider

Phase 3: Profiles & Policies

Phase 4: Budget & Truncation

Phase 5: React Integration

References

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions