awatch hits OS watch limits due to filtering occurring after watch registration #341
Description
Description
I'm using watchfiles as a dependency to sphinx-autobuild, and I've encountered an issue with how notifications are registered in very large directories. In my project, there is the equivalent of a .cache-directory with a very large number of files. Thus, when listening for changes in the repository, watchfiles tries to register watches on all files, leading to the following OS error:
OSError: OS file watch limit reached.
about ["/path/to/..."]
(Error { kind: MaxFilesWatch, paths: [...] })
Using the watch_filter is to no avail, as all the watches are still being registered, and the filtering only occurs after changes have been reported.
It would be great if the watch_files filter could be invoked prior to the watch registration - that would at least make it possible to manually avoid this issue for e.g. .cache-style directories.
Example Code
Watchfiles Output
....
raw-event=Access(Open(OSError: OS file watch limit reached. about ["@@@@@@/.direnv/flake-inputs/ninkaf94v6231yv0pyx401zfsw8pkxxx-source/pkgs/development/python-modules/lancedb"] (Error { kind: MaxFilesWatch, paths: ["@@@@@@/.direnv/flake-inputs/ninkaf94v6231yv0pyx401zfsw8pkxxx-source/pkgs/development/python-modules/lancedb"] })
Any)) event.kind=Access(Open(Any)) no change detected
.....
Operating System & Architecture
Linux-6.11.0-26-generic-x86_64-with-glibc2.40
#26~24.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Apr 17 19:20:47 UTC 2
Environment
No response
Python & Watchfiles Version
python: 3.12.10 (main, Apr 8 2025, 11:35:47) [GCC 14.2.1 20250322], watchfiles: 1.0.5
Rust & Cargo Version
No response