Skip to content

[Bug]: deferred image attachment loses its re-loadable reference in cached history; bot later denies seeing it #8151

Description

@singlerider

Affected component

channel

Severity

S1 - workflow blocked

Current behavior

When a user sends an image (e.g. a screenshot) and the bot is told to wait, the bot acks the message (thumbs-up reaction) but does not yet act on the image. On a later turn, when the user asks about the image, the bot has no record of it and denies any image was sent.

Root cause is in the channel orchestrator history cache. Inbound image markers [IMAGE:<path>] are reduced to a placeholder for cached/historical turns:

crates/zeroclaw-channels/src/orchestrator/mod.rs

fn channel_history_content_for_user_turn(content: &str) -> String {
    let (cleaned, image_refs) = zeroclaw_providers::multimodal::parse_image_markers(content);
    if image_refs.is_empty() {
        return content.to_string();
    }
    // ...
    if cleaned.is_empty() {
        "[Image attachment processed by vision model]".to_string()
    } else {
        cleaned
    }
}
fn strip_historical_image_payloads(turns: &mut Vec<ChatMessage>) {
    // ...
    for turn in &mut turns[..last_idx] {
        if turn.content.contains("[IMAGE:") {
            turn.content = channel_history_content_for_user_turn(&turn.content);
        }
    }
    // ...
}

The full [IMAGE:<path>] payload only survives for the current turn via restore_current_user_turn_media_payload. Once a newer message arrives, the image turn becomes historical and its marker is replaced by the literal text [Image attachment processed by vision model]. Two problems:

  1. The placeholder asserts the image was processed by a vision model even when it was only acked and deferred. It was never processed.
  2. The file-path reference is discarded, so no later turn can re-load the image from disk. The file itself still exists under the channel media dir (e.g. {workspace}/matrix_files/), but the handle to it is gone from history.

The strip logic does not distinguish a heavy inline base64 data: payload (which legitimately should not be re-sent every turn) from a cheap, re-loadable filesystem-path reference (which the vision route re-reads from disk on each turn).

Expected behavior

Historical image turns should retain a re-loadable reference (the [IMAGE:<path>] file-path marker) so a later turn can re-inflate and the vision route re-reads the file from disk. Only inline base64 data: URI payloads should be collapsed for historical turns, since those are heavy and the on-disk file is the source of truth. The bot should be able to answer "can you see the screenshot?" on a turn after the one that delivered it.

Steps to reproduce

1. Configure a vision-capable model_provider and a channel (observed on Matrix).
2. Send an image (screenshot) and tell the bot to wait.
3. Bot reacts thumbs-up (ack) and does not act on the image yet.
4. Send a follow-up: "can you see the screenshot?"
5. Bot denies any image was sent; the cached history shows only
   "[Image attachment processed by vision model]" with no path to re-load.

Impact

Affected users: anyone sending an image and following up on a later turn, or asking the bot to defer ("wait, look at this in a sec").
Frequency: always when the image turn is no longer the most recent turn.
Consequence: silent loss of the image reference; the bot confidently denies seeing an image the user clearly sent. The placeholder text is also misleading (claims vision processing that never happened).

ZeroClaw version

upstream/master @ 9efc7c0

Operating system

Linux (channel-agnostic; observed on Matrix)

Regression?

Unknown

Pre-flight checks

  • I reproduced this on the latest master branch or latest release.
  • I redacted secrets, tokens, and personal data from all submitted content.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingchannelAuto scope: src/channels/** changed.channel:matrixAuto module: channel/matrix changed.priority:p1High priorityproviderAuto scope: src/providers/** changed.runtimeAuto scope: src/runtime/** changed.

    Type

    No type

    Projects

    Status
    In Progress

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions