W[eb]FS

A FUSE (Filesystem in User SpacE) implementation that serves web resources (Public HTTP URLs, S3, etc.) as local files declared by minimal configs. While similar to and implementing the same protocol as the excellent rclone and proprietary cloud drive clients i.e. Google Drive & Dropbox, they are geared towards mounting/mirroring entire cloud drives locally with the cloud service centrally managing shared file names, directory trees, etc. The focus for WFS is on serving "links as files" so the entire internet can become your cloud provider, named and organized however you want just like any other on-device files.

Currently it supports HTTP(S) URL sources and a subset of system file operations (list + attributes, open, read ). See examples/pub_sources.json for how to define simple public URLs as files.

NOTE: WFS is in early, active development in my spare time, with a stable API still settling. If just curious and tinkering go for it expecting breaking changes, and only ever add reputable file sources you would physically download and open anyways.

Requirements

Linux, Mac, or FreeBSD(untested) operating system with appropriate os-specific FUSE binary installed and available on $PATH:

Linux

fusermount3 is available from all known major distro package managers ex. Arch Linux (btw): sudo pacman -S fuse3

MacOS

macfuse (formerly osxfuse) is recommended to be installed from their official releases

FreeBSD (untested)

Appears built-in??

Usage

go run ./cmd/wfs/main.go --nodes examples/pub_sources.json <mountpoint>

# In another shell (for now):
ls -l <mountpoint>
# Use like any other local file
<mpv|vlc|some-video-player> <mountpoint>/bbb/BigBuckBunny.mp4

See go run ./cmd/main.go --help for full cli args.

If something goes terribly wrong e.g. process killed without a chance to gracefully exit, try closing any processes that might be hanging on to open file handles (file explorer, terminal cwd in mount tree, etc) and fusermount -u <mountpoint> to unmount the filesystem.

Build

go build -o bin/wfs ./cmd/wfs/main.go

Status

Currently

TODO

• YouTube adapter & demo
• JSON5 support
• env var import, template & parse
• Refactor in-mem cache control to Filesystem
• Node meta write ops (mv, cp, ln, etc)
• Persistent Node defs
• Test FreeBSD
• "Magic File" add i.e. echo "https://example.com/file.txt" > /webfs/file.txt.wfs
• Hooks integration point (stdio "on-open", "on-update", "on-fail", "on-fetch", etc)
• Socket API
• Additional Protocol Providers
  • S3
  • FTP
  • SFTP
  • Google Drive
  • Dropbox
  • OneDrive
  • GCS
  • Azure Blob Storage
  • IPFS
• Lua? Adapters extension point

DONE

• Test MacOS
• Unit tests for initial read-only fs
• e2e testing pattern & basic MVP e2e tests
• Migrate & Iterate Python POC to Go MVP
  • Supports minimal list, open, attributes sys calls for public http/https urls
• Python POC