Any ideas on supporting Claude Code + Claude Code Router + Local LLM API (Ollama etc)?

[jccbbb]

Just wanna hear your thoughts 😊

[mistercrunch]

[mistercrunch]

@mistercrunch here answering from Agor through gh, here's Claude's answer ->

TL;DR: Yes! Two viable approaches to support local Ollama

Both achieve the same goal (local LLM support) via different technical paths:

---

Approach 1: OpenCode + Ollama ✅

OpenCode DOES support Ollama locally! Your semi-functional OpenCode integration can work.

How it works:

• OpenCode uses @ai-sdk/openai-compatible under the hood
• Ollama exposes OpenAI-compatible API at http://localhost:11434/v1
• Agor's existing OpenCode client connects to OpenCode server
• OpenCode server talks to local Ollama

Setup:

# 1. Fix Ollama's context window (CRITICAL - default 4k breaks tools!)
ollama pull qwen2.5-coder:32b
ollama run qwen2.5-coder:32b
>>> /set parameter num_ctx 32768
>>> /save qwen2.5-coder:32b-32k
>>> /bye

# 2. Configure OpenCode (~/.config/opencode/config.json)
{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "options": {
        "baseURL": "http://localhost:11434/v1"
      },
      "models": {
        "qwen2.5-coder:32b-32k": {
          "tools": true
        }
      }
    }
  }
}

# 3. Start OpenCode server
opencode serve --port 4096

Effort: ~1 week to polish existing integration

Pros:

• 75+ providers via OpenCode (Ollama, LM Studio, etc.)
• Advanced features (LSP, MCP)
• Future-proof

Cons:

• Requires OpenCode server running
• More moving parts

---

Approach 2: Claude Code Router via Env Vars ✅

Agor ALREADY has the infrastructure! I found that Agor already:

1. Supports per-user environment variables (encrypted in DB)
2. Merges them into process.env before calling Claude SDK
3. Claude Agent SDK respects ANTHROPIC_BASE_URL

How it works:

• User sets ANTHROPIC_BASE_URL in Agor UI settings
• Agor passes env vars to Claude Agent SDK subprocess
• SDK routes to local Ollama instead of Anthropic API

Setup:

# Terminal 1: Start Ollama
ollama serve
ollama pull qwen2.5-coder:32b

# In Agor UI: User Settings → Environment Variables
ANTHROPIC_BASE_URL=http://localhost:11434/v1
ANTHROPIC_AUTH_TOKEN=dummy

# Done! Claude sessions route to Ollama

Effort: ~3-5 days (UI polish + docs)

Pros:

• Infrastructure already exists (code in query-builder.ts:278-304)
• Fastest to ship
• Works with any OpenAI-compatible endpoint

Cons:

• Only works with OpenAI-compatible APIs
• Less flexible than OpenCode

---

Recommended Path

Ship Approach 2 first (env vars) for quick wins, then enhance Approach 1 (OpenCode) for power users.

Best Local Models (2025):

1. Qwen 2.5 Coder (32B) - GPT-4o level
2. Qwen 3 Coder - 256K context
3. DeepSeek Coder v2 - Strong generation
4. QwQ (32B) - Reasoning (needs 24GB RAM)

Critical Setup Tip:

Always increase Ollama's context window:

ollama run <model>
>>> /set parameter num_ctx 32768  # Default 4k breaks tool calling!
>>> /save <model>-32k

---

Question: Do you want:

• A) Quick path (env vars) - ships in days
• B) Robust path (OpenCode) - ships in 1-2 weeks
• C) Both (env vars now, OpenCode later)

My vote: C 🚀

[mistercrunch]

OpenCode support is super early/experimental, but I want to make it super well supported. It opens up TONS of cool models, and it's open source which is super great. Hoping to bring the integration to perfection (or as close to what the SDK will allow) over the next week or two.