Skip to content

Conversation

@Aki-07
Copy link

@Aki-07 Aki-07 commented Oct 18, 2025

What does this PR do?

This PR introduces a new community pipeline named VideoInpaintPipeline, which extends the existing StableDiffusionInpaintPipeline to support temporally-coherent video inpainting. The pipeline allows users to perform text-guided video editing and inpainting while maintaining temporal consistency across frames, enabling realistic motion, stable lighting, and coherent textures even on mid-tier GPUs.

It achieves this by:

  • Reusing diffusion noise and latent representations across consecutive frames
  • Optionally applying optical-flow-guided warping (RAFT) for motion-aware latent propagation
  • Adding batched FP16 inference, torch.compile acceleration, and streaming video I/O
  • Providing a clean, high-level API consistent with other Diffusers pipelines

Fixes #12477

Before submitting

Who can review?

@asomoza

@Aki-07 Aki-07 force-pushed the feat/add-community-video-pipeline branch from ff3ed09 to 1310cee Compare October 18, 2025 19:18
@Aki-07 Aki-07 force-pushed the feat/add-community-video-pipeline branch from 1310cee to 09abe15 Compare October 18, 2025 19:20
@Aki-07
Copy link
Author

Aki-07 commented Nov 3, 2025

@asomoza Requesting for review

@github-actions
Copy link
Contributor

github-actions bot commented Jan 9, 2026

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

@github-actions github-actions bot added the stale Issues that haven't received updates label Jan 9, 2026
@yiyixuxu yiyixuxu removed the stale Issues that haven't received updates label Jan 10, 2026
@@ -0,0 +1,62 @@
import numpy as np
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't need add tests here no? can you remove this?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the feedback, have removed the tests

@Aki-07
Copy link
Author

Aki-07 commented Jan 10, 2026

@yiyixuxu Thanks for the review! have addressed your feedback. Is there anything else needed from my side before this can be merged?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add VideoInpaintPipeline for temporally-consistent diffusion-based video inpainting

2 participants