Add motion-driven dynamic mask estimation (SEA-RAFT + Sampson)

[ladvu]

Adds an automatic, text-prompt-free alternative to TrackAnythingProcessor for
selecting dynamic objects in ViPE pipelines.

The new DynamicMaskProcessor runs auto-segmentation (SAM AutomaticMaskGenerator
+ DeAOT for ID-consistent tracking) on the input video, computes pairwise
optical flow with SEA-RAFT, derives a per-pixel Sampson reprojection error
against a RANSAC-fit fundamental matrix, and aggregates the error per tracked
instance to score how non-rigidly each instance moves. Instances above a
configurable fraction of the maximum motion score are classified as dynamic
and contribute to frame.mask. Selectable via the new motion_aware pipeline
config (vipe infer -p motion_aware ...).

Results on DAVIS:

[heiwang1997]

Hi @ladvu thank you so much for your contribution. This is a very interesting approach and super great that it allows to remove the requirement of predefined phrases.

Right now when I start to test it with the following command:

python run.py pipeline=motion_aware streams.base_path=../data/xxx.mp4 streams.frame_end=200 pipeline.slam.visualize=true

it fails with:

Traceback (most recent call last):
  File "/home/huangjh/workspace/.../vipe-public/run.py", line 23, in run
    pipeline.run(video_stream)
  File "/home/huangjh/workspace/.../vipe-public/vipe/pipeline/default.py", line 158, in run
    save_projection_video(
  File "/home/huangjh/workspace/.../vipe-public/vipe/utils/visualization.py", line 450, in save_projection_video
    img_iterators = [
  File "/home/huangjh/workspace/.../vipe-public/vipe/utils/visualization.py", line 451, in <listcomp>
    [
  File "/home/huangjh/workspace/.../vipe-public/vipe/utils/visualization.py", line 452, in <listcomp>
    {
KeyError: 'mask'

Do you mind testing this on your side and let me know what could go wrong?

[ladvu]

Hi @heiwang1997 thanks for your response. Will investigate this.

[ladvu]

@heiwang1997 Visualization now can be saved. Also export some useful artifacts. Do you mind test again?