Launch freezes on main thread inside RecursivePathWatcher.init → FSEventStreamStart when fseventsd is slow

[houko]

Summary

cmux freezes on launch (main thread blocked for 10+ seconds, up to minutes) whenever fseventsd is slow to accept new clients. The main thread is stuck inside RecursivePathWatcher.init → FSEventStreamStart → register_with_server → mach_msg, i.e. a synchronous Mach RPC to fseventsd performed on the main actor. FSEventStreamStart is not guaranteed to return promptly; when fseventsd is busy (rebuilding an out-of-sync event log on a slow external volume in my case, but Spotlight reindexing or Time Machine scans can do the same), every launch hangs until the daemon frees up.

Environment

• cmux 0.64.22 (build 102), stable channel
• macOS 27.0 (26A428), Apple Silicon
• Three external APFS volumes mounted; one of them (a WD My Passport SSD over USB) is very slow at UNMAP

What happened

Three consecutive launches hung today (10:40, 10:55, 11:02 JST). cmux's own hang detector captured all three; the main-thread stack is identical in each sample, 100% of samples on this frame chain:

2315 Thread_30221   DispatchQueue_1: com.apple.main-thread  (serial)
+ 2315 completeTaskWithClosure(swift::AsyncContext*, swift::SwiftError*)  (in libswift_Concurrency.dylib)
+   ...
+             2315 RecursivePathWatcher.init(paths:clock:)  (in cmux) + 936
+               2315 ???  (in cmux)  load address 0x100388000 + 0x21e7f30
+                 2315 FSEventStreamStart  (in FSEvents) + 260
+                   2315 register_with_server  (in FSEvents) + 864
+                     2315 f2d_register_rpc  (in FSEvents) + 284
+                       2315 mach_msg  (in libsystem_kernel.dylib) + 24
+                         2315 mach_msg_overwrite
+                           2315 mach_msg2_internal
+                             2315 mach_msg2_trap

(stallSeconds=9.044 in the hang metadata is just when the detector sampled; the actual freeze lasted until the user force-quit.)

The unified log shows why fseventsd was not answering. The external volume had been unmounted uncleanly, so on remount fseventsd decided its event log was out of sync and started rebuilding it on that volume, which at the same moment was crawling through a 14-minute mount-time APFS TRIM:

10:53:20.156 fseventsd  event logs ... out of sync with volume.  destroying old logs.
10:53:20.156 fseventsd  event post failed for <private>
11:03:05.245 fseventsd  log dir: <private> getting new uuid: <private>      ← 10 minutes later
11:03:08.048 fseventsd  fsevent_add_client: no bundle id available for pid 3333   ← queued clients finally admitted
11:03:08.103 fseventsd  fsevent_add_client: no bundle id available for pid 3065
11:03:08.104 fseventsd  fsevent_add_client: no bundle id available for pid 2726
11:03:08.104 fseventsd  fsevent_add_client: no bundle id available for pid 1865

kernel  10:53:50  disk11 tx xid 1706727 took 26302574 us to close
kernel  11:07:14  spaceman_trim_free_blocks: disk11 scan took 842.997765 s, trims took 841.179491 s

Between 10:53:20 and 11:03:05 no process could register a new FSEvents stream. The other queued pids were CLIs without a UI, so nobody noticed them; cmux was the only one blocking its main thread on it. Once fseventsd recovered, the next cmux launch (11:05) came up normally with no changes on my side.

Where it happens

• Packages/macOS/CmuxFoundation/Sources/CmuxFoundation/FileWatch/RecursivePathWatcher.swift (main @ 47a2860): the doc comment at line 25 states the design intent, "The FSEventStream is created synchronously in init, so the watcher is already listening when it returns", and line 132 constructs FileSystemEventStream inline in init.
• Packages/macOS/CmuxFoundation/Sources/CmuxFoundation/FileWatch/FileSystemEventStream.swift line 139: guard FSEventStreamStart(stream) else { ... } inside init?.
• Packages/macOS/CmuxSidebarGit/Sources/CmuxSidebarGit/Service/SidebarGitMetadataService+Watchers.swift line 108: SidebarGitMetadataService (declared @MainActor, SidebarGitMetadataService.swift line 25) calls RecursivePathWatcher(paths:throttleInterval:eventFilter:) directly, so the whole chain runs on the main thread.

The "already listening when init returns" guarantee is what makes the main actor pay for fseventsd's latency. FSEventStreamCreate is cheap and local; FSEventStreamStart is the call that talks to the daemon, and that is the one that blocks.

Suggested fix

Keep the no-gap guarantee but move the daemon round-trip off the main actor:

1. Create and start the stream on a background executor (a detached task or the watcher's own dispatch queue) and hand the actor a ready watcher when FSEventStreamStart returns, instead of doing it inline in a synchronous init that @MainActor callers can reach.
2. Because the watcher already has a requiresFullRescan overflow marker, the "missed events during the gap" concern is coverable: emit one requiresFullRescan batch as soon as the stream is live, so any change that happened between the request and the start is picked up by the existing conservative re-check path.
3. Optionally bound the wait: if FSEventStreamStart has not returned after a few seconds, mark the watcher degraded and fall back to the existing polling path until it does. That keeps the git sidebar functional (if slower) instead of freezing the window.

The same pattern applies to any other @MainActor site that constructs FileSystemEventStream synchronously.

How to reproduce

The reliable way is to make fseventsd busy and then launch cmux with at least one workspace whose git watcher will be created at startup:

1. Mount an external APFS volume with a lot of fragmented free space on a slow-UNMAP USB SSD, unmount it uncleanly (pull the cable or reboot without ejecting), plug it back in. log stream --predicate 'process == "fseventsd"' will show destroying old logs and then nothing for a while.
2. Launch cmux while that is in progress. The window never paints; sample cmux shows the stack above.

A cheaper approximation for development: suspend the daemon (sudo kill -STOP $(pgrep -x fseventsd)), launch cmux, observe the freeze, then kill -CONT. That is artificial, but it exercises exactly the same blocking path.

Hang samples and the full log show excerpts are available if useful.