Skip to content

fix(daemon): seed Claude image attachments (MUL-3907)#2877

Open
YOMXXX wants to merge 2 commits into
multica-ai:mainfrom
YOMXXX:codex/claude-image-first-message
Open

fix(daemon): seed Claude image attachments (MUL-3907)#2877
YOMXXX wants to merge 2 commits into
multica-ai:mainfrom
YOMXXX:codex/claude-image-first-message

Conversation

@YOMXXX

@YOMXXX YOMXXX commented May 19, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes Claude-backed daemon tasks not reliably understanding image attachments. The root cause is that Claude was asked to download/read issue images through the CLI/Read tool path, which can degrade into a text-only tool_result flow. This PR puts image attachments into Claude's initial user message as native image content blocks while keeping the existing CLI attachment download path as fallback/context.

Related Issue

Closes #2400

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Refactor / code improvement (no behavior change)
  • Tests (adding or improving test coverage)
  • Documentation update
  • CI / infrastructure

Changes Made

  • server/internal/handler/daemon.go: include issue attachment metadata in daemon task claims.
  • server/internal/daemon/client.go: add daemon-side attachment metadata/download helper, matching the existing signed-url download behavior.
  • server/internal/daemon/daemon.go: for Claude runtimes, download eligible image attachments and pass up to 4 images (max 10 MB each) into execution options.
  • server/pkg/agent/claude.go: encode those images as Claude JSONL initial user-message image blocks.
  • Added tests for claim metadata, daemon image hydration, attachment download behavior, and Claude input encoding.

How to Test

  1. cd server && go test ./pkg/agent ./internal/daemon ./internal/handler -count=1
  2. Create or use an issue with an image attachment and assign it to a Claude runtime.
  3. Confirm the daemon claim includes issue_attachments and Claude receives native image blocks in the initial message.

Checklist

  • I have included a thinking path that traces from project context to this change
  • I have run tests locally and they pass
  • I have added or updated tests where applicable
  • If this change affects the UI, I have included before/after screenshots
  • I have updated relevant documentation to reflect my changes
  • If I added a new runtime / coding tool / UI tab, I synced the change to landing copy (apps/web/features/landing/i18n/), starter-content (packages/views/onboarding/utils/starter-content-content-*.ts), and relevant docs (apps/docs/content/docs/)
  • If this PR touches Chinese product copy, I checked it against apps/docs/content/docs/developers/conventions.zh.mdx (terminology, mixed-rule for task / issue / skill)
  • I have considered and documented any risks above
  • I will address all reviewer comments before requesting merge

AI Disclosure

AI tool used: Codex

Prompt / approach: Used a systematic debugging path: traced #2400 from issue/task claims through daemon prompt construction and Claude JSONL input, identified the local-path/Read-tool image path as the failing boundary, then added a minimal native image-block path for Claude with focused Go tests.

Screenshots (optional)

N/A - daemon/backend change only.

@vercel

vercel Bot commented May 19, 2026

Copy link
Copy Markdown

Someone is attempting to deploy a commit to the IndexLabs Team on Vercel.

A member of the Team first needs to authorize it.

@YOMXXX
YOMXXX force-pushed the codex/claude-image-first-message branch 2 times, most recently from 37d28c6 to e8eca27 Compare May 22, 2026 16:36
@YOMXXX
YOMXXX force-pushed the codex/claude-image-first-message branch 2 times, most recently from be72043 to 89049b6 Compare May 31, 2026 09:22
@YOMXXX
YOMXXX force-pushed the codex/claude-image-first-message branch from 89049b6 to f751f25 Compare June 19, 2026 15:16
@YOMXXX

YOMXXX commented Jun 19, 2026

Copy link
Copy Markdown
Contributor Author

Rebased this Claude image attachment seeding fix onto current upstream/main and force-pushed with lease: f751f250c.

Conflict resolution kept newer main behavior and layered the image path on top:

  • preserved multi-message chat prompt aggregation and collected attachments from every unanswered user message;
  • preserved thread names, quick-create attachment IDs, parent/sub-issue fields, task-scoped auth, openclaw mode, and resume/workdir guardrails;
  • added issue-level attachment metadata to claims, unified claim attachment metadata as AttachmentMeta, and pass eligible image attachments into Claude as initial image blocks.

Local validation:

  • go test ./pkg/agent -run 'TestBuildClaudeInput|TestClaudeExecute|TestClaudeArgsForLog|TestBuildClaudeArgs|TestApplyClaude|TestTrySendDropsWhenFull' -count=1 -v
  • go test ./internal/daemon -count=1
  • go test ./internal/handler -run TestClaimTask_IncludesIssueAttachments -count=1 -v (handler DB tests skipped locally because Postgres is not running)
  • git diff --check upstream/main...HEAD

CI is running on the new head now.

@YOMXXX

YOMXXX commented Jun 19, 2026

Copy link
Copy Markdown
Contributor Author

CI is green on the rebased head f751f250c: frontend, backend, installer (ubuntu-latest), and installer (macos-latest) all completed successfully. Merge state is now CLEAN.

@Codefreyy

Copy link
Copy Markdown

Hey, just came across this while tracking #2400. The fix looks solid — CI is green, rebase is clean, and the author has been very responsive. Is there anything the community can do to help push this forward? Would be great to see it merged.

@Bohan-J Bohan-J left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tackling #2400 — seeding the image as a native content block up front is the right direction, and the relative-vs-absolute download auth split plus the encoding tests are clean. A few blocking issues before this can go in, though.

Blocking

1. The attachment download can't authenticate → the feature is a silent no-op

Client.DownloadAttachment (server/internal/daemon/client.go:609-611) first fetches metadata from GET /api/attachments/{id}. That route is registered under middleware.Auth + middleware.RequireWorkspaceMember (see server/cmd/server/router.go), which needs a workspace to be resolvable from the request. But:

  • middleware.Auth only stamps X-Workspace-ID for mat_ task tokens; for the daemon's PAT / cloud-PAT it sets X-User-ID only.
  • The daemon client never sends X-Workspace-Slug / X-Workspace-ID nor a workspace_id query param.

So ResolveWorkspaceIDFromRequest returns empty and the request is rejected with 400 "workspace_id is required" before any bytes are served. claudeInitialImages swallows that as a Warn and continues (server/internal/daemon/daemon.go:3101-3104), so no image is ever embedded — first run or resume. The unit tests pass only because they mock the endpoint with httptest and bypass the real middleware.

Every other workspace-scoped daemon call goes through /api/daemon/... (under DaemonAuth); this is the first daemon call to a member-scoped route, which is why it slips through in tests but fails in a real deployment.

Suggested fix: call the existing GET /api/attachments/{id}/download instead. It's auth-only, self-resolves the workspace from the attachment row, and enforces membership internally (the runtime owner is a workspace member, so it passes). That also drops the separate metadata round-trip — content_type is already in the claim's AttachmentMeta. Alternatively, add a /api/daemon/.../attachments/{id} route under DaemonAuth.

2. Media-type filter is too broad → some images crash the turn

claudeInitialImages (daemon.go:3098 and :3114) accepts anything with an image/ prefix, but the Anthropic API only supports image/jpeg, image/png, image/gif, image/webp. On upload, .svg is rewritten to image/svg+xml (extContentTypes), and http.DetectContentType classifies .bmp / .ico as image/bmp / image/x-icon — all of these pass the filter but get 400'd by the API, which fails the entire initial user message and the task. That's a regression for issues carrying those attachment types. Please restrict to the supported set and skip the rest.

3. Images are re-injected on every resumed turn

runTask sets execOpts.InitialImages unconditionally for Claude (daemon.go:3559) regardless of task.PriorSessionID. On a resumed session the images are already in the transcript, so re-embedding them every turn re-downloads the bytes and piles up duplicate copies in context (N copies after N turns). Gate on cold start, e.g. if provider == "claude" && task.PriorSessionID == "" { ... }.

Nits

  • Download-before-size-check: downloadFile reads up to 100 MB (client.go:652) and the 10 MB cap is only applied afterward (daemon.go:3106), so a 50 MB image is fully downloaded and then discarded. size_bytes is available server-side (and the claim handler already reads the row), so plumbing it into AttachmentMeta (types.go:119) would let you skip oversized images without downloading them. Related: the loop keeps downloading until it collects 4 images, so it isn't bounded to 4 downloads when early attachments are oversized.
  • buildClaudeInput: the inner if image.MediaType != "" guard is dead — an empty MediaType is already continued at claude.go:622.
  • downloadFile truncates silently at 100 MB+1 rather than erroring on oversize. Fine for images, but a bit rough.
  • The branch currently conflicts with main (daemon.go / agent.go / daemon_test.go) — needs a rebase.

Happy to re-review once #1 and #2 are addressed. Thanks again!

@Bohan-J Bohan-J changed the title fix(daemon): seed Claude image attachments fix(daemon): seed Claude image attachments (MUL-3907) Jul 1, 2026
@YOMXXX
YOMXXX force-pushed the codex/claude-image-first-message branch from f751f25 to 43a0965 Compare July 7, 2026 07:09
@YOMXXX

YOMXXX commented Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the blocking review items in 43a0965e5 and force-pushed the rebased branch:

  • attachment downloads now use /api/attachments/{id}/download directly instead of the member-scoped metadata route;
  • initial Claude image seeding is restricted to image/jpeg, image/png, image/gif, and image/webp;
  • initial images are only seeded for cold Claude tasks, not resumed sessions.

Local verification:

  • go test ./internal/daemon -run 'TestClient_DownloadAttachmentUsesDownloadEndpointWithAuthHeaders|TestClaudeInitialImagesDownloadsImageAttachments|TestClaudeInitialImagesSkipsUnsupportedImageTypes|TestShouldSeedClaudeInitialImagesOnlyForColdClaudeTasks' -count=1
  • go test ./internal/daemon
  • go test ./pkg/agent -run 'TestBuildClaudeInput|TestClaudeExecute|TestClaudeArgsForLog|TestBuildClaudeArgs|TestApplyClaude|TestTrySendDropsWhenFull' -count=1
  • GOPROXY=https://goproxy.cn,direct go test ./internal/handler -run TestClaimTask_IncludesIssueAttachments -count=1
  • git diff --check

CI is green on the updated head.

@YOMXXX
YOMXXX force-pushed the codex/claude-image-first-message branch from 43a0965 to 544c175 Compare July 8, 2026 14:57
@YOMXXX

YOMXXX commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current upstream/main and cleared the dirty merge state. Conflict resolution preserved the current chat-intro/task fields from main while keeping the attachment metadata shape needed for Claude image seeding: issue attachments and chat message attachments still carry id, filename, and content_type; cold Claude tasks seed only supported image types and resumed sessions skip initial image seeding. Local verification: - cd server && go test ./internal/daemon -run 'TestClient_DownloadAttachmentUsesDownloadEndpointWithAuthHeaders|TestClaudeInitialImagesDownloadsImageAttachments|TestClaudeInitialImagesSkipsUnsupportedImageTypes|TestShouldSeedClaudeInitialImagesOnlyForColdClaudeTasks|TestBuildChatPromptAttachmentIDsCanBeBoundToCreatedIssues' -count=1 - cd server && go test ./internal/handler -run 'TestClaimTask_IncludesIssueAttachments|TestClaimTaskByRuntime_ChatIntroGateClearsAfterUserReplies' -count=1 - cd server && go test ./pkg/agent -run 'TestBuildClaudeInputEncodesInitialImages' -count=1 - git diff --check The earlier blocking review items remain addressed; ready for re-review.

@YOMXXX

YOMXXX commented Jul 12, 2026

Copy link
Copy Markdown
Contributor Author

Re-review request: re-verified the three blocking Claude image attachment items on the current head 544c17514.

Current behavior:

  • attachment bytes are fetched via /api/attachments/{id}/download, not the member-scoped metadata endpoint;
  • initial seeding only accepts Anthropic-supported image media types;
  • initial images are seeded only for cold Claude tasks, not resumed sessions.

Verification run today:

  • cd server && go test ./internal/daemon -run 'TestClient_DownloadAttachmentUsesDownloadEndpointWithAuthHeaders|TestClaudeInitialImagesDownloadsImageAttachments|TestClaudeInitialImagesSkipsUnsupportedImageTypes|TestShouldSeedClaudeInitialImagesOnlyForColdClaudeTasks|TestBuildChatPromptAttachmentIDsCanBeBoundToCreatedIssues' -count=1\n- cd server && go test ./internal/handler -run 'TestClaimTask_IncludesIssueAttachments|TestClaimTaskByRuntime_ChatIntroGateClearsAfterUserReplies' -count=1\n- cd server && go test ./pkg/agent -run 'TestBuildClaudeInputEncodesInitialImages' -count=1\n- git diff --check\n\nCI is green; ready for re-review.

@YOMXXX
YOMXXX force-pushed the codex/claude-image-first-message branch from 544c175 to 9011539 Compare July 12, 2026 09:04
@YOMXXX

YOMXXX commented Jul 12, 2026

Copy link
Copy Markdown
Contributor Author

Follow-up: rebased this PR onto current upstream/main and force-pushed 90115394a to clear the dirty merge state.

Conflict resolution kept the current main chat input-owner / coalesced-comment claim flow and preserved this PR's attachment metadata surface:

  • issue_attachments still carries attachment id, filename, and content_type;
  • chat message attachments still use the same AttachmentMeta shape;
  • Claude initial image seeding remains limited to cold tasks and supported media types.

Verification after rebase:

  • cd server && go test ./internal/daemon -run 'TestClient_DownloadAttachmentUsesDownloadEndpointWithAuthHeaders|TestClaudeInitialImagesDownloadsImageAttachments|TestClaudeInitialImagesSkipsUnsupportedImageTypes|TestShouldSeedClaudeInitialImagesOnlyForColdClaudeTasks|TestBuildChatPromptAttachmentIDsCanBeBoundToCreatedIssues' -count=1\n- cd server && go test ./internal/handler -run 'TestClaimTask_IncludesIssueAttachments|TestClaimTaskByRuntime_ChatIntroGateClearsAfterUserReplies' -count=1\n- cd server && go test ./pkg/agent -run 'TestBuildClaudeInputEncodesInitialImages' -count=1\n- git diff --check

@YOMXXX
YOMXXX force-pushed the codex/claude-image-first-message branch from 9011539 to ac48944 Compare July 19, 2026 12:11
@YOMXXX

YOMXXX commented Jul 19, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current upstream/main and force-pushed ac4894488; the PR is now merge-clean again.

The earlier blocking review items remain addressed on the rebased head:

  • attachment bytes are fetched via /api/attachments/{id}/download;
  • Claude initial image seeding is limited to supported media types (image/jpeg, image/png, image/gif, image/webp);
  • initial images are only seeded for cold Claude tasks, not resumed sessions.

Verification after rebase:

  • cd server && go test ./internal/daemon -run 'TestClient_DownloadAttachmentUsesDownloadEndpointWithAuthHeaders|TestClaudeInitialImagesDownloadsImageAttachments|TestClaudeInitialImagesSkipsUnsupportedImageTypes|TestShouldSeedClaudeInitialImagesOnlyForColdClaudeTasks|TestBuildChatPromptAttachmentIDsCanBeBoundToCreatedIssues' -count=1
  • cd server && go test ./internal/handler -run 'TestClaimTask_IncludesIssueAttachments|TestClaimTaskByRuntime_ChatIntroGateClearsAfterUserReplies' -count=1
  • cd server && go test ./pkg/agent -run 'TestBuildClaudeInputEncodesInitialImages' -count=1
  • git diff --check

GitHub CI is green across changes, backend, frontend, and installers. Ready for re-review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Images cannot be recognized by the agent

3 participants