Skip to content

remotemount.refresh: live mount updates without unmounting#3330

Open
cpuhrsch wants to merge 1 commit intometa-pytorch:mainfrom
cpuhrsch:export-D98953121
Open

remotemount.refresh: live mount updates without unmounting#3330
cpuhrsch wants to merge 1 commit intometa-pytorch:mainfrom
cpuhrsch:export-D98953121

Conversation

@cpuhrsch
Copy link
Copy Markdown
Contributor

@cpuhrsch cpuhrsch commented Mar 31, 2026

Summary:
Add MountHandler.refresh(sourcepath) for atomically updating FUSE mount
data while the filesystem stays mounted. Open file handles remain valid
and subsequent reads see the new data immediately.

Walkthrough

Rust (chunked_fuse.rs): Extract FsData struct (metadata + chunks +
chunk_size) and wrap in Arc<RwLock> shared between ChunkedFuseFs
and PyMountHandle. All FUSE methods acquire a read lock (non-blocking
with each other). PyMountHandle.refresh() acquires the write lock and
atomically swaps the data. TTL is set to zero so the kernel always
re-validates metadata after refresh.

Rust (tls_sender.rs): Fix pre-existing flake where tcp
shutdown(Write) failed with ENOTCONN when the receiver closed first.
This is benign (data already sent) — now ignored instead of fatal.

Python (remotemount.py): FUSEActor.mount() now mounts an empty FUSE
filesystem and calls _do_refresh() to populate it, sharing the
flush/cache/swap logic with FUSEActor.refresh_mount(). MountHandler uses
a shared _sync() method for open() and refresh() that packs the source
directory, diffs block hashes, transfers dirty blocks, then mounts or
refreshes. refresh(sourcepath) requires the caller to pass the source
path again to prevent accidentally refreshing with a forgotten path.

Tests: 5 FUSE unit tests (TestFuseRefresh) covering data refresh,
metadata refresh, open file handles across refresh, repeated refreshes,
and end-to-end pack+refresh+read. 4 integration tests with actors
covering refresh, open handles, no-change, and add-file scenarios.

Benchmark: bench_fuse_read.py measures FUSE read latency and
throughput vs tmpfs baseline.

Reviewed By: zdevito

Differential Revision: D98953121

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Mar 31, 2026
@meta-codesync
Copy link
Copy Markdown

meta-codesync bot commented Mar 31, 2026

@cpuhrsch has exported this pull request. If you are a Meta employee, you can view the originating Diff in D98953121.

Summary:
Add MountHandler.refresh(sourcepath) for atomically updating FUSE mount
data while the filesystem stays mounted. Open file handles remain valid
and subsequent reads see the new data immediately.

## Walkthrough

**Rust (chunked_fuse.rs):** Extract FsData struct (metadata + chunks +
chunk_size) and wrap in Arc<RwLock<FsData>> shared between ChunkedFuseFs
and PyMountHandle. All FUSE methods acquire a read lock (non-blocking
with each other). PyMountHandle.refresh() acquires the write lock and
atomically swaps the data. TTL is set to zero so the kernel always
re-validates metadata after refresh.

**Rust (tls_sender.rs):** Fix pre-existing flake where tcp
shutdown(Write) failed with ENOTCONN when the receiver closed first.
This is benign (data already sent) — now ignored instead of fatal.

**Python (remotemount.py):** FUSEActor.mount() now mounts an empty FUSE
filesystem and calls _do_refresh() to populate it, sharing the
flush/cache/swap logic with FUSEActor.refresh_mount(). MountHandler uses
a shared _sync() method for open() and refresh() that packs the source
directory, diffs block hashes, transfers dirty blocks, then mounts or
refreshes. refresh(sourcepath) requires the caller to pass the source
path again to prevent accidentally refreshing with a forgotten path.

**Tests:** 5 FUSE unit tests (TestFuseRefresh) covering data refresh,
metadata refresh, open file handles across refresh, repeated refreshes,
and end-to-end pack+refresh+read. 4 integration tests with actors
covering refresh, open handles, no-change, and add-file scenarios.

**Benchmark:** bench_fuse_read.py measures FUSE read latency and
throughput vs tmpfs baseline.

Reviewed By: zdevito

Differential Revision: D98953121
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot. fb-exported meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant