Sandbox sync: git-based sync strategy and E2B real-time sync (descoped from #76)

[frankbria]

Context

Issue #76 (Phase 6.3 Sandbox File Synchronization) implemented snapshot-sync filtering for the E2B provider: .ralphignore support, --sync-include/--sync-exclude, large-file policy, progress summaries, selective artifact retrieval, and deletion-sync safety (see docs/SANDBOX_SYNC.md). Two of its original acceptance criteria were deliberately descoped (owner-approved during implementation):

Descoped items

1. Git-based sync strategy

Use git push/pull as the sync mechanism between host and sandbox.

Why deferred: it directly conflicts with the reviewed #75 safety model — .git is excluded from sync in both directions so sandbox-side git state can never clobber the host repository — and requires git credentials inside the sandbox (a new secret-handling surface). Adopting it means revisiting that design first.

2. Real-time sync for E2B

Continuous bidirectional file watching (inotify/fswatch + sandbox daemon).

Why deferred: the per-iteration download already lands changes at every loop boundary, which is the natural sync point for an autonomous loop; a watcher adds conflict-resolution complexity for little gain. Docker already has real-time sync by architecture (rw bind mount).

References

• [P4] Phase 6.3 Sandbox File Synchronization #76 (parent), [P4] Phase 6.0 Sandbox execution environments (E2B, Docker, and others) #49 (Phase 6 umbrella), [P4] Phase 6.1 Local Docker Sandbox Execution #74 (Docker), [P4] Phase 6.2 E2B Cloud Sandbox Integration #75 (E2B)
• docs/SANDBOX_SYNC.md "Out of scope" section

[github-actions]

Triage: Feature request / enhancement

Categorized this as an enhancement (already labeled enhancement + phase-6, so no label change is needed).

Reasoning

• This is a deliberate, owner-authored follow-up that tracks the two acceptance criteria descoped from [P4] Phase 6.3 Sandbox File Synchronization #76 during implementation: (1) git-based sync strategy and (2) E2B real-time sync. It is forward-looking proposed work, not a bug.
• Not a duplicate. [P4] Phase 6.3 Sandbox File Synchronization #76 (the Phase 6.3 umbrella) is still open, but Sandbox sync: git-based sync strategy and E2B real-time sync (descoped from #76) #304 narrowly scopes the deferred subset that [P4] Phase 6.3 Sandbox File Synchronization #76 left out. Closing it against [P4] Phase 6.3 Sandbox File Synchronization #76 would lose that scoping, so it should stay open as a distinct enhancement.

Recommended next steps

• Keep this open as the tracking issue for the deferred sync strategies; link it from [P4] Phase 6.3 Sandbox File Synchronization #76 so the parent reflects what was descoped.
• Before adopting the git-based strategy, revisit the [P4] Phase 6.2 E2B Cloud Sandbox Integration #75 safety model — .git is currently excluded from sync both directions specifically so sandbox-side git state can't clobber the host repo, and credentials-in-sandbox is a new secret-handling surface (coordinate with the Phase 6.4 security work).
• For E2B real-time sync, weigh the conflict-resolution complexity against the existing per-iteration download, which already lands changes at every loop boundary.
• Consider a priority: label to slot it relative to the rest of Phase 6 (the parent [P4] Phase 6.3 Sandbox File Synchronization #76 sits at P4).

No assignment made — this is a planning/tracking enhancement rather than an actionable bug.

Generated by Issue Triage Assistant for issue #304 · opus48 528.2K · ◷

[coderabbitai]

Coding Plan

Summary

• Phase 1 establishes git credential infrastructure as a foundation, treating credentials as opt-in due to their broader access scope
• Phase 2 implements git-based sync using bundle upload and patch-based download, preserving the safety invariant that sandbox commits cannot directly modify host git state
• Phase 3 adds optional real-time sync with explicit conflict handling, complementing rather than replacing the per-iteration sync model
• All new sync modes are opt-in via explicit flags, maintaining backward compatibility with existing tar-based snapshot sync

Design Choices

Design Choice 1: Git-based sync coexistence with existing safety model

Options Considered:

1. Full .git directory sync with commit-level merge on download (complex conflict resolution)
2. Git bundle upload with detached worktree in sandbox, patch-based download (no sandbox commits affect host)
3. Shared bare repository (requires external git remote, additional infrastructure)

Chosen Option: 2

Rationale: Option 2: Git bundle upload with patch-based download preserves the safety invariant that sandbox commits cannot directly modify host git state while enabling git tooling inside the sandbox.

Design Choice 2: Real-time sync trigger mechanism for E2B

Options Considered:

1. Polling-based sync (periodic find -newer calls, higher latency)
2. Sandbox-side inotify daemon with host callback endpoint (requires exposing host endpoint)
3. Sandbox-side watcher pushing to host via E2B SDK file operations with host-side polling receiver

Chosen Option: 3

Rationale: Option 3: Sandbox-side watcher with SDK-based push and host-side polling receiver avoids exposing host endpoints while providing near-real-time sync within E2B's operational model.

💡 User Tips

Regenerate the plan with different choices with @coderabbitai <feedback>.

Implementation Steps

Phase 1: Git Credential Infrastructure for Sandboxes

Enable secure forwarding of git credentials to sandbox environments, required as a foundation for git-based sync operations.

Task 1: Define Git Credential Forwarding Strategy

Design the credential types and forwarding mechanisms supported for sandbox git operations.

• Extend lib/sandbox_e2b.sh credential seeding (_seed_e2b_claude_credentials) to support optional git credentials
• Support two credential modes: SSH key forwarding (mount ~/.ssh/id_* read-only) and HTTPS token via git credential helper
• Add --sandbox-git-credentials CLI flag to ralph_loop.sh argument parser (disabled by default for safety)
• Document security implications: git credentials have broader access than Claude API keys

Task 2: Implement E2B Git Credential Seeding

Add git credential injection to the E2B sandbox initialization flow.

• Create _seed_e2b_git_credentials() in lib/sandbox_e2b.sh following the pattern of _seed_e2b_claude_credentials()
• For SSH mode: write host SSH keys to /home/user/.ssh/ with 0600 permissions, configure known_hosts
• For HTTPS mode: configure git credential helper with token via git config --global credential.helper
• Credentials should be seeded after sandbox creation in start_e2b_sandbox()

Task 3: Implement Docker Git Credential Forwarding

Extend Docker credential setup to optionally mount git credentials.

• Update setup_docker_credentials() in lib/sandbox_docker.sh to handle git credential mounting
• Add volume mount for ~/.ssh (read-only) when SSH mode enabled
• For HTTPS mode: seed credential helper config into $SANDBOX_CLAUDE_HOME/.gitconfig
• Ensure credentials are cleaned up in cleanup_docker_sandbox()

🤖 Prompt for AI agents

Implement git credential infrastructure for sandbox environments (E2B and
Docker). The goal is to allow optional, secure forwarding of git credentials as
a foundation for git-based sync.

**`lib/sandbox_e2b.sh`:**
- Extend `_seed_e2b_claude_credentials` to support optional git credentials
- Create a new `_seed_e2b_git_credentials()` function following the pattern of
`_seed_e2b_claude_credentials()`
- SSH mode: write host `~/.ssh/id_*` keys to `/home/user/.ssh/` with `0600`
permissions and configure known_hosts
- HTTPS mode: configure git credential helper with token via `git config
--global credential.helper`
- Call `_seed_e2b_git_credentials()` after sandbox creation in
`start_e2b_sandbox()`

**`ralph_loop.sh`:**
- Add `--sandbox-git-credentials` CLI flag to the argument parser (disabled by
default for safety)
- Document security implications: git credentials have broader access than
Claude API keys

**`lib/sandbox_docker.sh`:**
- Update `setup_docker_credentials()` to handle git credential mounting
  - SSH mode: add a read-only volume mount for `~/.ssh`
- HTTPS mode: seed credential helper config into
`$SANDBOX_CLAUDE_HOME/.gitconfig`
- Ensure credentials are cleaned up in `cleanup_docker_sandbox()`

Phase 2: Git-Based Sync for E2B

Replace tar-based snapshot sync with git bundle upload and patch-based download, preserving host repository safety.

Task 1: Git Bundle Upload Implementation

Replace tarball upload with git bundle for initial project sync to sandbox.

• Add _build_git_bundle_for_upload() function in lib/sandbox_e2b.sh that creates a git bundle containing all refs
• Bundle should include working tree state as an uncommitted stash or temporary commit
• Modify upload_project_to_e2b() to use bundle when SANDBOX_SYNC_MODE=git (new env var, default: tar)
• Add cmd_upload_git subcommand to lib/e2b_helper.py that clones the bundle and checks out the working state

Task 2: Git-Based Change Detection in Sandbox

Replace find -newer change detection with git-based diff tracking.

• Modify cmd_download in lib/e2b_helper.py to support git mode
• Use git diff --name-only HEAD to find changed files (both staged and unstaged)
• Generate unified diff patches instead of raw file contents for transmission
• Maintain manifest generation for deletion tracking (existing pattern)

Task 3: Patch-Based Download and Application

Apply sandbox changes to host as patches rather than direct file overwrites.

• Create _apply_git_patches() function in lib/sandbox_e2b.sh as alternative to tar extraction
• Apply patches using git apply --3way for conflict detection
• On conflict: preserve both versions with conflict markers, log warning, continue sync
• Update sync_e2b_artifacts_down() to dispatch to git or tar mode based on SANDBOX_SYNC_MODE

Task 4: Sandbox Git State Isolation

Ensure sandbox git operations cannot affect host repository integrity.

• Sandbox-side commits remain local to sandbox (no push back to host's .git)
• Host receives only working-tree changes as patches
• Add explicit validation in _apply_git_patches() that refuses to modify .git/ directory
• Document that sandbox can commit for its own workflow but those commits are ephemeral

🤖 Prompt for AI agents

Implement git-based sync for E2B, replacing the tar-based snapshot mechanism
with git bundle upload and patch-based download. The goal is to enable git
tooling inside the sandbox while preserving the host repository safety
invariant.

**`lib/sandbox_e2b.sh`:**
- Add `_build_git_bundle_for_upload()` that creates a git bundle containing all
refs, including working tree state as an uncommitted stash or temporary commit
- Modify `upload_project_to_e2b()` to branch on `SANDBOX_SYNC_MODE` env var
(new, default: `tar`): use bundle upload when set to `git`
- Create `_apply_git_patches()` as an alternative to tar extraction:
  - Apply patches using `git apply --3way` for conflict detection
- On conflict: preserve both versions with conflict markers, log a warning, and
continue sync
  - Add explicit validation that refuses to modify `.git/` directory
- Update `sync_e2b_artifacts_down()` to dispatch to git or tar mode based on
`SANDBOX_SYNC_MODE`

**`lib/e2b_helper.py`:**
- Add `cmd_upload_git` subcommand that clones the uploaded bundle and checks out
the working state
- Modify `cmd_download` to support git mode:
  - Use `git diff --name-only HEAD` to find changed files (staged and unstaged)
  - Generate unified diff patches instead of raw file contents for transmission
  - Maintain existing manifest generation for deletion tracking

Phase 3: E2B Real-Time Sync Infrastructure

Add continuous bidirectional file synchronization between host and E2B sandbox using file watchers.

Task 1: Sandbox-Side File Watcher Daemon

Implement an inotify-based watcher that runs inside the E2B sandbox.

• Create templates/e2b_file_watcher.py daemon script that monitors /home/user/workspace
• Use inotifywait or Python watchdog library for file change detection
• On change: write changed file paths to /tmp/ralph_realtime_changes (append-only log)
• Exclude .git/ and .ralph/ from monitoring
• Daemon should be started after upload_project_to_e2b() via _e2b_helper exec

Task 2: Host-Side Realtime Sync Orchestration

Add background sync polling during Claude execution.

• Create _start_realtime_sync_poller() in lib/sandbox_e2b.sh that spawns a background process
• Poller reads /tmp/ralph_realtime_changes from sandbox at configurable interval (default 2s)
• For each changed path: download and extract individual file, similar to existing download flow
• Add --sandbox-realtime-sync flag to enable (disabled by default)

Task 3: Host-to-Sandbox Change Propagation

Enable host file changes to propagate to sandbox during execution.

• Add host-side file watcher using fswatch or inotifywait (if available) or polling fallback
• On host change: upload individual changed files via _e2b_helper write-file
• Coordinate with sandbox watcher to avoid echo loops (use temp marker files)
• Only activate when --sandbox-realtime-sync is enabled

Task 4: Conflict Detection and Resolution

Handle simultaneous changes to the same file from both host and sandbox.

• Implement last-writer-wins with backup: on conflict, preserve losing version as .file.conflict
• Log all conflicts to .ralph/logs/sync_conflicts.log with timestamps and paths
• Add SANDBOX_SYNC_CONFLICT_MODE env var: last-writer-wins (default), host-wins, sandbox-wins
• Document conflict scenarios and resolution behavior in sync documentation

🤖 Prompt for AI agents

Implement real-time bidirectional file synchronization infrastructure for E2B
sandboxes. The goal is to provide continuous, near-real-time file sync between
host and sandbox using file watchers, as an opt-in feature.

**`templates/e2b_file_watcher.py`** (new file):
- Create a daemon script that monitors `/home/user/workspace`
- Use `inotifywait` or the Python `watchdog` library for file change detection
- On change: append changed file paths to `/tmp/ralph_realtime_changes`
(append-only log)
- Exclude `.git/` and `.ralph/` from monitoring
- Daemon is started after `upload_project_to_e2b()` via `_e2b_helper exec`

**`lib/sandbox_e2b.sh`:**
- Create `_start_realtime_sync_poller()` that spawns a background process:
- Reads `/tmp/ralph_realtime_changes` from sandbox at a configurable interval
(default: 2s)
- For each changed path: download and extract the individual file using the
existing download flow
- Add host-side file watcher using `fswatch` or `inotifywait` (with polling
fallback):
  - On host change: upload individual changed files via `_e2b_helper write-file`
- Use temp marker files to coordinate with the sandbox watcher and avoid echo
loops
- Implement conflict resolution logic:
- Last-writer-wins with backup: preserve the losing version as `.file.conflict`
- Log all conflicts to `.ralph/logs/sync_conflicts.log` with timestamps and
paths
- Read `SANDBOX_SYNC_CONFLICT_MODE` env var to determine resolution mode:
`last-writer-wins` (default), `host-wins`, `sandbox-wins`

**`ralph_loop.sh`:**
- Add `--sandbox-realtime-sync` flag to enable real-time sync (disabled by
default)

Phase 4: Documentation and Configuration

Update documentation and add configuration options for new sync modes.

Task 1: Update E2B Sandbox Documentation

Document git-based sync and real-time sync features.

• Update docs/E2B_SANDBOX.md with new sync modes section
• Document SANDBOX_SYNC_MODE, --sandbox-git-credentials, --sandbox-realtime-sync options
• Add security considerations section for git credential forwarding
• Document conflict resolution behavior and configuration

Task 2: Create Sync Strategy Guide

Create dedicated documentation for sync strategy selection.

• Create docs/SANDBOX_SYNC.md covering all sync strategies (tar, git, realtime)
• Document tradeoffs: tar (simple, safe), git (preserves history, requires credentials), realtime (low latency, complex)
• Include decision flowchart for choosing sync strategy
• Document per-iteration sync (default) vs realtime sync use cases

🤖 Prompt for AI agents

Write and update documentation for the new sync modes and configuration options
introduced in Phases 1–3.

**`docs/E2B_SANDBOX.md`:**
- Add a new sync modes section documenting `SANDBOX_SYNC_MODE`,
`--sandbox-git-credentials`, and `--sandbox-realtime-sync` options
- Add a security considerations section covering git credential forwarding
- Document conflict resolution behavior and the `SANDBOX_SYNC_CONFLICT_MODE`
configuration

**`docs/SANDBOX_SYNC.md`** (new file):
- Cover all sync strategies: tar (simple, safe), git (preserves history,
requires credentials), realtime (low latency, complex)
- Document tradeoffs for each strategy
- Include a decision flowchart for choosing a sync strategy
- Document per-iteration sync (default) vs. realtime sync use cases

Research

The E2B sandbox (lib/sandbox_e2b.sh, lib/e2b_helper.py) currently uses a snapshot-based sync model: project files are uploaded once at startup via tarball, and changed files are downloaded after each Claude iteration using a find -newer marker mechanism. The .git directory is explicitly excluded in both directions—upload uses git ls-files (which never lists .git internals) and download uses tar --exclude='.git' flags. No git credentials are forwarded to sandboxes; only Claude API credentials are seeded. No file-watching infrastructure exists anywhere in the codebase. Docker sandbox uses bind mounts, requiring no sync layer at all.

---

🚀 Next Steps

🤖 All AI agent prompts combined

Task: 1

Implement git credential infrastructure for sandbox environments (E2B and
Docker). The goal is to allow optional, secure forwarding of git credentials as
a foundation for git-based sync.

**`lib/sandbox_e2b.sh`:**
- Extend `_seed_e2b_claude_credentials` to support optional git credentials
- Create a new `_seed_e2b_git_credentials()` function following the pattern of
`_seed_e2b_claude_credentials()`
- SSH mode: write host `~/.ssh/id_*` keys to `/home/user/.ssh/` with `0600`
permissions and configure known_hosts
- HTTPS mode: configure git credential helper with token via `git config
--global credential.helper`
- Call `_seed_e2b_git_credentials()` after sandbox creation in
`start_e2b_sandbox()`

**`ralph_loop.sh`:**
- Add `--sandbox-git-credentials` CLI flag to the argument parser (disabled by
default for safety)
- Document security implications: git credentials have broader access than
Claude API keys

**`lib/sandbox_docker.sh`:**
- Update `setup_docker_credentials()` to handle git credential mounting
  - SSH mode: add a read-only volume mount for `~/.ssh`
- HTTPS mode: seed credential helper config into
`$SANDBOX_CLAUDE_HOME/.gitconfig`
- Ensure credentials are cleaned up in `cleanup_docker_sandbox()`
===============================================================================

Task: 2

Implement git-based sync for E2B, replacing the tar-based snapshot mechanism
with git bundle upload and patch-based download. The goal is to enable git
tooling inside the sandbox while preserving the host repository safety
invariant.

**`lib/sandbox_e2b.sh`:**
- Add `_build_git_bundle_for_upload()` that creates a git bundle containing all
refs, including working tree state as an uncommitted stash or temporary commit
- Modify `upload_project_to_e2b()` to branch on `SANDBOX_SYNC_MODE` env var
(new, default: `tar`): use bundle upload when set to `git`
- Create `_apply_git_patches()` as an alternative to tar extraction:
  - Apply patches using `git apply --3way` for conflict detection
- On conflict: preserve both versions with conflict markers, log a warning, and
continue sync
  - Add explicit validation that refuses to modify `.git/` directory
- Update `sync_e2b_artifacts_down()` to dispatch to git or tar mode based on
`SANDBOX_SYNC_MODE`

**`lib/e2b_helper.py`:**
- Add `cmd_upload_git` subcommand that clones the uploaded bundle and checks out
the working state
- Modify `cmd_download` to support git mode:
  - Use `git diff --name-only HEAD` to find changed files (staged and unstaged)
  - Generate unified diff patches instead of raw file contents for transmission
  - Maintain existing manifest generation for deletion tracking
===============================================================================

Task: 3

Implement real-time bidirectional file synchronization infrastructure for E2B
sandboxes. The goal is to provide continuous, near-real-time file sync between
host and sandbox using file watchers, as an opt-in feature.

**`templates/e2b_file_watcher.py`** (new file):
- Create a daemon script that monitors `/home/user/workspace`
- Use `inotifywait` or the Python `watchdog` library for file change detection
- On change: append changed file paths to `/tmp/ralph_realtime_changes`
(append-only log)
- Exclude `.git/` and `.ralph/` from monitoring
- Daemon is started after `upload_project_to_e2b()` via `_e2b_helper exec`

**`lib/sandbox_e2b.sh`:**
- Create `_start_realtime_sync_poller()` that spawns a background process:
- Reads `/tmp/ralph_realtime_changes` from sandbox at a configurable interval
(default: 2s)
- For each changed path: download and extract the individual file using the
existing download flow
- Add host-side file watcher using `fswatch` or `inotifywait` (with polling
fallback):
  - On host change: upload individual changed files via `_e2b_helper write-file`
- Use temp marker files to coordinate with the sandbox watcher and avoid echo
loops
- Implement conflict resolution logic:
- Last-writer-wins with backup: preserve the losing version as `.file.conflict`
- Log all conflicts to `.ralph/logs/sync_conflicts.log` with timestamps and
paths
- Read `SANDBOX_SYNC_CONFLICT_MODE` env var to determine resolution mode:
`last-writer-wins` (default), `host-wins`, `sandbox-wins`

**`ralph_loop.sh`:**
- Add `--sandbox-realtime-sync` flag to enable real-time sync (disabled by
default)
===============================================================================

Task: 4

Write and update documentation for the new sync modes and configuration options
introduced in Phases 1–3.

**`docs/E2B_SANDBOX.md`:**
- Add a new sync modes section documenting `SANDBOX_SYNC_MODE`,
`--sandbox-git-credentials`, and `--sandbox-realtime-sync` options
- Add a security considerations section covering git credential forwarding
- Document conflict resolution behavior and the `SANDBOX_SYNC_CONFLICT_MODE`
configuration

**`docs/SANDBOX_SYNC.md`** (new file):
- Cover all sync strategies: tar (simple, safe), git (preserves history,
requires credentials), realtime (low latency, complex)
- Document tradeoffs for each strategy
- Include a decision flowchart for choosing a sync strategy
- Document per-iteration sync (default) vs. realtime sync use cases

💡 Iterate on the plan with: `@coderabbitai` <feedback>

Example Feedback
- `@coderabbitai` You can skip phase 3. Add a simple unit test case for phase 2.
- `@coderabbitai` For design choice 1 go ahead with option 3 and replan.

---

💬 Have feedback or questions? Drop into our discord!

[coderabbitai]

🔗 Related PRs

#276 - docs(readme): sync feature documentation with implemented feature set (#82) [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[frankbria]

Closing as not planned — these two items were deliberately descoped from #76, and the reasons documented there still hold:

• Git-based sync conflicts with the .git-exclusion safety model from [P4] Phase 6.2 E2B Cloud Sandbox Integration #75 (sandbox-side git state must never be able to clobber the host repository) and would require git credentials inside the sandbox — a new secret-handling surface.
• Real-time sync for E2B duplicates what the per-iteration download already provides: changes land at every loop boundary, which is the natural sync point for an autonomous loop. Docker has real-time sync by architecture (rw bind mount).

With Docker (#74) + E2B (#75) shipped plus sync filtering (#76), ralph's sandbox feature set is complete (see docs/SANDBOX_SYNC.md "Out of scope"). Richer execution environments are CodeFrame territory.

A community PR implementing this against the documented constraints would still be reviewed (bug-fix-sized changes only).