Support virtio-fs #88

MikazukiHikari · 2025-07-02T16:01:16Z

Overview

This commit introduces the VirtIO file system device (virtio-fs) to enable efficient and secure sharing of host directories with semu.

Implementation Details

Virtio-fs is implemented as a VIRTIO device in the emulator, following the FUSE protocol for all file system requests.
The guest acts as a FUSE client, while semu acts as the FUSE server, handling requests such as :
- INIT
- GETATTR
- OPENDIR
- READDIRPLUS
- LOOKUP
- FORGET
- RELEASEDIR
- OPEN
- READ
- RELEASE
- FLUSH
- DESTROY
Each request from the guest is placed into a virtqueue and processed by semu, which performs the actual filesystem operations and returns the results.
In this implementation, a hash table (inode_map, using uthash.h) is used to efficiently map guest inode numbers to host file paths, ensuring fast lookups and consistent file operations. If we don't utilize it, which may results lacking an efficient and unique mapping structure increases the risk of losing, duplicating, or corrupting inode-path associations, leading to abnormal file operations, data loss, or security issues.
The device supports mounting with a tag, e.g.:

semu# mount -t virtiofs myfs /mnt

Test Procedures

Build and launch the emulator with virtio-fs enabled.
On the guest, mount the shared directory:

mount -t virtiofs myfs /mnt

Verify file operations:

Read files in the shared directory in host.
Support commands like ls, cat, and cd to ensure correct operation.

Unmount and remount to confirm stability.

umount /mnt

This project is implemented on 6.11.0-26-generic 24.04.1-Ubuntu GNU/Linux.

jserv · 2025-07-03T00:29:50Z

README.md

@@ -77,13 +77,33 @@ You can exit the emulator using: \<Ctrl-a x\>. (press Ctrl+A, leave it, afterwar
 ## Usage

 ```shell
-./semu -k linux-image [-b dtb-file] [-i initrd-image] [-d disk-image]
+./semu -k linux-image [-b dtb-file] [-i initrd-image] [-d disk-image] [-s mount-folder]


Never use "folder." Instead, use "directory."

jserv · 2025-07-03T00:30:11Z

configs/linux.config

@@ -102,7 +102,8 @@ CONFIG_SRCU=y
 CONFIG_TINY_SRCU=y
 # end of RCU Subsystem

-# CONFIG_IKCONFIG is not set
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y


Minimize the necessary changes.

jserv · 2025-07-03T00:30:21Z

configs/linux.config

@@ -1257,7 +1262,7 @@ CONFIG_INOTIFY_USER=y
 # CONFIG_QUOTA is not set
 CONFIG_AUTOFS4_FS=y
 CONFIG_AUTOFS_FS=y
-# CONFIG_FUSE_FS is not set
+


Don't do this.

jserv

Fix build breakage.

shengwen-tw · 2025-07-03T03:09:58Z

fuse.h

+    int32_t error;
+    uint64_t unique;
+};
+


Remove redundant line.

shengwen-tw · 2025-07-03T03:11:08Z

fuse.h

+
+struct fuse_lookup_in {
+    uint64_t parent;  // inode of parent dir
+    // char name[];     // followed by name (not null-terminated!)


Don't leave dead code. Add TODO if this requires further development.

shengwen-tw · 2025-07-03T03:11:48Z

main.c

+
+#if SEMU_HAS(VIRTIOFS)
+        case 0x48: /* virtio-fs */
+            // printf("width:%d\n",width);


Remove debugging code.

shengwen-tw · 2025-07-03T03:12:26Z

minimal.dts

+            compatible = "virtio,mmio";
+            reg = <0x4800000 0x1000>;
+            interrupts = <6>;
+            tag = "myfs";


Is tag necessary?

shengwen-tw · 2025-07-03T03:12:47Z

virtio.h

+#define FUSE_RELEASE 18
+#define FUSE_FLUSH 25
+#define FUSE_DESTROY 38
+


Remove redundant line.

shengwen-tw · 2025-07-03T03:14:37Z

fuse.h

+};
+
+struct fuse_init_in {
+    uint32_t major;  // FUSE major version supported by the guest (typically 7)


Follow the coding style guide from CONTRIBUTING.md, i.e., using /* ... */ instead of // ... for single-line comment.

shengwen-tw · 2025-07-03T03:15:24Z

fuse.h

+    uint32_t major;          // FUSE major version supported by the device
+    uint32_t minor;          // FUSE minor version supported by the device
+    uint32_t max_readahead;  // Maximum readahead size accepted by the device
+    uint32_t


Rewrite like

/* Flags supported by the device (negotiated with the guest) */ uint32_t flags;

to avoid line breaking.

shengwen-tw · 2025-07-03T03:17:39Z

fuse.h

+    uint64_t lock_owner;
+};
+
+struct fuse_flush_in {


Wrap with PACKED() for all structures that require proper memory alignment.

Check the following code as example:

semu/common.h

Line 31 in 5cb78df

#define PACKED(name) name __attribute__((packed))

semu/virtio.h

Line 60 in 5cb78df

PACKED(struct virtq_desc {

shengwen-tw · 2025-07-03T03:18:39Z

virtio-fs.c

+#include <sys/uio.h>
+#include <time.h>
+#include <unistd.h>
+


Remove all redundant lines in the file.

shengwen-tw · 2025-07-03T03:21:10Z

virtio-fs.c

+    case FUSE_DESTROY:
+        virtio_fs_destroy_handler(vfs, vq_desc, plen);
+        break;
+    default:


Add TODO for the OP codes to implement in the future.

jserv

Always write comments in C-style and American English.

shengwen-tw · 2025-07-03T03:30:18Z

Clean up the commit history by squashing related commits using

$ git rebase -i GIT_HASH

jserv · 2025-07-03T03:30:35Z

Why uhash.h is used? Clarify its necessity.

shengwen-tw · 2025-07-03T03:33:48Z

Please improve the pull request description to make the content clearer and easier to understand. See:
Check: #70

shengwen-tw · 2025-07-03T05:06:24Z

Please move configs/linux.config to a separated commit for better history tracking.

MikazukiHikari · 2025-07-04T08:44:41Z

Many thanks to @shengwen-tw .

I rebased the branch, and split the changes into two commits: one for configs/linux.config, and the other for the rest.
Besides, I improved the pull request description to make the content clearer and easier to understand, which includes the necessity of uthash.h.
Please examine the changes at your convenience.

shengwen-tw · 2025-07-04T15:21:42Z

device.h

+
+typedef struct {
+    uint64_t ino;
+    char path[4096];


Instead of hard-coding the value, consider using PATH_MAX from the standard library (limits.h) for better portability and readability.

#include <stdio.h> #include <limits.h> int main(void) { printf("PATH_MAX is defined: %d\n", PATH_MAX); return 0; }

shengwen-tw · 2025-07-04T15:22:12Z

main.c

+
+#if SEMU_HAS(VIRTIOFS)
+        case 0x48: /* virtio-fs */
+            // printf("width:%d\n",width);


Remove debugging code.

shengwen-tw · 2025-07-04T15:24:19Z

main.c

-        execpath);
+    fprintf(stderr,
+            "Usage: %s -k linux-image [-b dtb] [-i initrd-image] [-d "
+            "disk-image] [-s mount-folder]\n",


Use directory (UNIX terminology) instead of folder (Windows terminology).

shengwen-tw · 2025-07-04T15:25:36Z

virtio.h

@@ -24,6 +24,45 @@
 #define VIRTIO_BLK_S_IOERR 1
 #define VIRTIO_BLK_S_UNSUPP 2

+#define FUSE_INIT 26


Could you point out the source for the FUSE OP codes and what are missing in the current implementation?

shengwen-tw · 2025-07-04T15:26:16Z

device.h

+    /* guest memory base */
+    uint32_t *ram;
+
+    char *mount_tag;  // guest sees this tag


Use /* ... */ instead.

shengwen-tw · 2025-07-04T15:27:41Z

virtio-fs.c

+PACKED(struct virtio_fs_config {
+    char tag[36];
+    uint32_t num_request_queues;
+    uint32_t notify_buf_size;  // ignored


Rewrite all // ... with /* ... */ in this file.

shengwen-tw · 2025-07-04T15:28:25Z

virtio-fs.c

+
+typedef struct {
+    DIR *dir;
+    char path[4096];


Is it possible to use malloc() instead of a fixing path size?

shengwen-tw · 2025-07-04T15:28:55Z

virtio-fs.c

+    uint32_t *ram = vfs->ram;
+    void *priv = vfs->priv;
+    char *mount_tag = vfs->mount_tag;
+    char shared_dir[4096];


Is it possible to use malloc() instead of a fixing path size?

shengwen-tw · 2025-07-04T15:30:45Z

virtio-fs.c

+    *plen = header_resp->out.len;
+}
+
+


Remove redundant lines.

shengwen-tw · 2025-07-04T15:33:10Z

virtio-fs.c

+        fprintf(stderr, "unsupported virtio-fs operation!\n");
+        return -1;
+    }
+    /* TO DO: FUSE_WRITE, FUSE_MKDIR, FUSE_RMDIR	, FUSE_CREATE */


Use TODO instead of TO DO.
Check Effectively Managing Technical Debt with TODO, FIXME, and Other Code Reminders.

shengwen-tw · 2025-07-04T15:37:51Z

README.md

@@ -77,13 +77,33 @@ You can exit the emulator using: \<Ctrl-a x\>. (press Ctrl+A, leave it, afterwar
 ## Usage

 ```shell
-./semu -k linux-image [-b dtb-file] [-i initrd-image] [-d disk-image]
+./semu -k linux-image [-b dtb-file] [-i initrd-image] [-d disk-image] [-s mount-directory]


mount-directory might be ambiguous, especially in contexts involving virtio-blk devices. Consider renaming it to something more descriptive, such as shared-directory.

shengwen-tw · 2025-07-04T15:42:00Z

Drop Finally, use clang-format-18 to fit the code style. in the commit message.

ChinYikMing · 2025-07-05T10:08:59Z

@MikazukiHikari Hi, could you provide a Linux image with virtio-fs enabled for quick testing? Thank you!

ChinYikMing · 2025-07-05T10:10:28Z

Please rebase to the latest remote master branch. Otherwise, the build on macOS will fail.

MikazukiHikari · 2025-07-05T11:14:32Z

@MikazukiHikari您好，能否提供一個啟用了 virtio-fs 的 Linux 鏡像，方便我們快速測試一下？謝謝！

No problem! How should I send it to you? via Gmail? Or just commit the Linux image?

ChinYikMing · 2025-07-05T11:22:37Z

@MikazukiHikari您好，能否提供一個啟用了 virtio-fs 的 Linux 鏡像，方便我們快速測試一下？謝謝！

No problem! How should I send it to you? via Gmail? Or just commit the Linux image?

You can simply upload to this code review thread.

MikazukiHikari · 2025-07-05T11:37:26Z

Image.zip

MikazukiHikari · 2025-07-05T11:38:16Z

@MikazukiHikari您好，能否提供一個啟用了 virtio-fs 的 Linux 鏡像，方便我們快速測試一下？謝謝！

No problem! How should I send it to you? via Gmail? Or just commit the Linux image?

You can simply upload to this code review thread.

THX for the reminder.

ChinYikMing · 2025-07-05T14:19:10Z

Verify file operations:
Create, read, write, and delete files in the shared directory in host.

@MikazukiHikari After a quick test, it seems that the create operation is not yet implemented. However, you mentioned file creation in your earlier comment.

I encountered the following error when trying to create a file using the touch command in the guest:

Welcome to Buildroot
buildroot login: root
# mount -t virtiofs myfs /mnt
# touch /mnt/apple
unsupported virtio-fs operation!

My testing environment:
OS: macOS/arm64 Version 15.3.1 (Darwin 24.3.0 Darwin Kernel Version 24.3.0)
Compiler: Apple clang version 16.0.0 (clang-1600.0.26.6)

MikazukiHikari · 2025-07-05T16:07:16Z

Verify file operations:
Create, read, write, and delete files in the shared directory in host.

@MikazukiHikari After a quick test, it seems that the create operation is not yet implemented. However, you mentioned file creation in your earlier comment.

I encountered the following error when trying to create a file using the touch command in the guest:
Welcome to Buildroot
buildroot login: root
# mount -t virtiofs myfs /mnt
# touch /mnt/apple
unsupported virtio-fs operation!
My testing environment: OS: macOS/arm64 Version 15.3.1 (Darwin 24.3.0 Darwin Kernel Version 24.3.0) Compiler: Apple clang version 16.0.0 (clang-1600.0.26.6)

You're right! I wrote the previous comment incorrectly — it's been corrected. Sorry for the confusion.
BTW, touch command is about FUSE_CREATE, and it is not yet implemented.

Map MMIO region at 0xF48____ for virtio-fs device in semu. Introduce special-case logic for char tag in virtio_fs config handling in semu. Add inode_map hash table for mapping inodes to file paths. Implement FUSE core operations: - INIT - GETATTR - OPENDIR - READDIRPLUS - LOOKUP - FORGET - RELEASEDIR - OPEN - READ - RELEASE - FLUSH - DESTROY to make semu supports commands:`cd`, `cat`, `ls`. Introduced uthash.h to implement a hash table for mapping inode numbers to file paths. This improves lookup performance and simplifies path management in the virtio-fs implementation. Set default mount tag to "myfs" and mount directory to "./shared". When `make check`, if MOUNT_DIRECTORY is not exist, then create a new one. If -s parameter is empty, virtio-fs is unused. Add the explanation about virtio-fs in `Usage` of README.md. Add the introduction of how to mount and unmount in semu.

jserv changed the title ~~Implement virtio filesystem in semu~~ Support virtio-fs Jul 3, 2025

jserv requested a review from shengwen-tw July 3, 2025 00:29

jserv reviewed Jul 3, 2025

View reviewed changes

jserv requested changes Jul 3, 2025

View reviewed changes

shengwen-tw reviewed Jul 3, 2025

View reviewed changes

jserv requested changes Jul 3, 2025

View reviewed changes

MikazukiHikari force-pushed the master branch from be8566e to fb4c14a Compare July 3, 2025 05:01

MikazukiHikari force-pushed the master branch from fb4c14a to 52d21c0 Compare July 3, 2025 05:31

MikazukiHikari force-pushed the master branch from 52d21c0 to a757556 Compare July 4, 2025 15:13

shengwen-tw reviewed Jul 4, 2025

View reviewed changes

MikazukiHikari force-pushed the master branch 2 times, most recently from 9168250 to 9210af1 Compare July 4, 2025 18:52

Update linux.config for virtio-fs

99960e6

MikazukiHikari force-pushed the master branch from 9210af1 to a58d09e Compare July 5, 2025 16:11

Support virtio-fs #88

Are you sure you want to change the base?

Support virtio-fs #88

Uh oh!

Conversation

MikazukiHikari commented Jul 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Implementation Details

Test Procedures

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jserv left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shengwen-tw Jul 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jserv left a comment

Choose a reason for hiding this comment

Uh oh!

shengwen-tw commented Jul 3, 2025

Uh oh!

jserv commented Jul 3, 2025

Uh oh!

shengwen-tw commented Jul 3, 2025

Uh oh!

shengwen-tw commented Jul 3, 2025

Uh oh!

MikazukiHikari commented Jul 4, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shengwen-tw Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shengwen-tw commented Jul 4, 2025

MikazukiHikari commented Jul 2, 2025 •

edited

Loading

shengwen-tw Jul 3, 2025 •

edited

Loading

shengwen-tw Jul 4, 2025 •

edited

Loading

MikazukiHikari commented Jul 5, 2025 •

edited

Loading

ChinYikMing commented Jul 5, 2025 •

edited

Loading

MikazukiHikari commented Jul 5, 2025 •

edited

Loading