Skip to content

ZeroGPU native support#13316

Open
cbensimon wants to merge 21 commits intomainfrom
queue-multiprocess
Open

ZeroGPU native support#13316
cbensimon wants to merge 21 commits intomainfrom
queue-multiprocess

Conversation

@cbensimon
Copy link
Copy Markdown
Collaborator

@cbensimon cbensimon commented Apr 22, 2026


Note

Medium Risk
Adds conditional ZeroGPU middleware and new multiprocessing-aware queue message routing, which can affect request handling and progress/log delivery on Spaces. Risk is moderate due to concurrency/process-boundary behavior changes gated by environment flags.

Overview
Adds native ZeroGPU support by conditionally installing spaces.zero.ZeroGPUMiddleware when running in a ZeroGPU Space, including mapping ZeroGPU errors into Gradio Errors and emitting logs via Gradio’s logger.

Introduces MultiprocessWorkerContextualizer to refresh LocalContext in persistent worker processes and updates queue progress/log messaging to optionally forward updates from child processes back to the main server process when GRADIO_QUEUE_MULTIPROCESSING_ENABLED=true.

Also removes legacy spaces.gradio_auto_wrap wrapping of BlockFunction, makes server start timeout configurable via GRADIO_START_TIMEOUT, and updates test/deps tooling to include spaces>=0.50.dev0 (and avoid pin exclusion for spaces).

Reviewed by Cursor Bugbot for commit 6bb3b3b. Bugbot is set up for automated code reviews on this repo. Configure here.

@gradio-pr-bot
Copy link
Copy Markdown
Collaborator

gradio-pr-bot commented Apr 22, 2026

🪼 branch checks and previews

Name Status URL
Spaces ready! Spaces preview
Website building...
🦄 Changes detected! Details

Install Gradio from this PR

pip install https://huggingface.co/buckets/gradio/pypi-previews/resolve/6bb3b3b5e236ffa8e9db8cc15028491d05dbae77/gradio-6.14.0-py3-none-any.whl

Install Gradio Python Client from this PR

pip install "gradio-client @ git+https://github.com/gradio-app/gradio@6bb3b3b5e236ffa8e9db8cc15028491d05dbae77#subdirectory=client/python"

Install Gradio JS Client from this PR

npm install https://gradio-npm-previews.s3.amazonaws.com/6bb3b3b5e236ffa8e9db8cc15028491d05dbae77/gradio-client-2.2.0.tgz

@gradio-pr-bot
Copy link
Copy Markdown
Collaborator

gradio-pr-bot commented Apr 22, 2026

🦄 change detected

This Pull Request includes changes to the following packages.

Package Version
gradio minor

  • ZeroGPU native support

✅ Changeset approved by @freddyaboulton

  • Maintainers can remove approval by unchecking this checkbox.

Something isn't right?

  • Maintainers can change the version label to modify the version bump.
  • If the bot has failed to detect any changes, or if this pull request needs to update multiple packages to different versions or requires a more comprehensive changelog entry, maintainers can update the changelog file directly.

@freddyaboulton
Copy link
Copy Markdown
Collaborator

This approach looks good to me @cbensimon

@cbensimon
Copy link
Copy Markdown
Collaborator Author

This approach looks good to me @cbensimon

For context I am working on sibling PR:

I'll add the ZeroGPUMiddleware registration part once ready on spaces side

PS: spaces will still contain a gradio "compat mode" that registers the middleware + patches Gradio queue for older Gradio versions

Keeping you updated!

@cbensimon cbensimon changed the title feat(queue): multiprocessing support ZeroGPU native support May 4, 2026
@cbensimon cbensimon marked this pull request as ready for review May 4, 2026 16:12
Comment thread gradio/queueing.py
Comment thread gradio/context.py
Comment thread gradio/queueing.py Outdated
@cbensimon cbensimon requested a review from freddyaboulton May 4, 2026 17:39
Copy link
Copy Markdown
Collaborator

@freddyaboulton freddyaboulton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good @cbensimon ! Tried to test myself with spaces 0.50.0 dev version but the spaces image is hardcoding version 0.48

Comment thread scripts/create_test_requirements.sh Outdated
Comment thread gradio/context.py
Comment thread gradio/utils.py Outdated
Comment thread gradio/blocks.py Outdated
Comment thread gradio/queueing.py
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

There are 3 total unresolved issues (including 1 from previous review).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit df84193. Configure here.

Comment thread gradio/routes.py Outdated
Comment thread gradio/context.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants