Allow applying gr.cache() to intermediate functions directly

[abidlabs]

Now you can do gr.cache() within your function to cache an intermediate function, like this:

import time
import gradio as gr

with gr.Blocks() as demo:
    def expensive_lookup(query):
        time.sleep(2)
        return f"Results for {query}"

    def event(query):
        cached_lookup = gr.cache(expensive_lookup)
        results = cached_lookup(query)
        return f"Final response based on: {results}"

    t = gr.Textbox()
    t.submit(event, t, t)

demo.launch()

[gradio-pr-bot]

🪼 branch checks and previews

•	Name	Status	URL
	Spaces	ready!	Spaces preview
	Website	ready!	Website preview
🦄	Changes	detected!	Details

---

Install Gradio from this PR

pip install https://huggingface.co/buckets/gradio/pypi-previews/resolve/5ae37bc6e5f68f4531693efb67e7c1c485e48e3b/gradio-6.13.0-py3-none-any.whl

Install Gradio Python Client from this PR

pip install "gradio-client @ git+https://github.com/gradio-app/gradio@5ae37bc6e5f68f4531693efb67e7c1c485e48e3b#subdirectory=client/python"

Install Gradio JS Client from this PR

npm install https://gradio-npm-previews.s3.amazonaws.com/5ae37bc6e5f68f4531693efb67e7c1c485e48e3b/gradio-client-2.2.0.tgz

[gradio-pr-bot]

🦄 change detected

This Pull Request includes changes to the following packages.

Package	Version
gradio	minor

---

• Allow applying gr.cache() to intermediate functions directly

---

‼️ Changeset not approved. Ensure the version bump is appropriate for all packages before approving.

• Maintainers can approve the changeset by checking this checkbox.

Something isn't right?

• Maintainers can change the version label to modify the version bump.
• If the bot has failed to detect any changes, or if this pull request needs to update multiple packages to different versions or requires a more comprehensive changelog entry, maintainers can update the changelog file directly.

[Copilot]

Pull request overview

Adds support for calling gr.cache() at runtime (e.g., gr.cache(fn)(...)) so intermediate helper calls inside a larger Gradio callback can reuse the same cache store and properly report cache usage.

Changes:

• Implemented a runtime wrapper registry so repeated cache(fn, ...) calls reuse the same wrapper/store and can mark cache hits for “used cache” UI reporting.
• Added pytest coverage for runtime cache(fn)(...) usage (reuse, key function support, manual-cache usage tracking, and error cases).
• Updated the caching guide and added a changeset entry documenting the new capability.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File	Description
gradio/caching.py	Adds wrapper caching/registry and hit tracking to support runtime cache(fn)(...).
test/test_caching.py	Adds tests validating runtime wrapper reuse, key behavior, manual-cache tracking, and runtime misuse errors.
guides/04_additional-features/17_caching.md	Documents caching intermediate helper calls via runtime gr.cache(fn)(...).
.changeset/bright-stars-smoke.md	Declares a minor release for the new caching feature.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

_cache_wrappers is a process-global dict that only ever grows; each unique (func,key,max_size,max_memory,per_session) combination will be retained for the lifetime of the process along with its _CacheStore. In long-running servers, runtime caching of dynamically-created callables (e.g., inner helpers created per request) can lead to unbounded memory growth. Consider bounding/evicting the wrapper registry (LRU), providing an explicit clear mechanism, and/or using weak references keyed by callable identity so wrappers for short-lived callables can be garbage-collected.

Suggested change

	# Runtime `gr.cache(fn)(...)` may be evaluated on every request, so keep one
	# shared wrapper/store per function+config instead of recreating an empty cache.
	_cache_wrappers: dict[tuple[Any, ...], Callable] = {}
	# Requests can resolve `gr.cache(fn)` concurrently; guard registry creation so
	class _WrapperRegistry(OrderedDict[tuple[Any, ...], Callable]):
	def __init__(self, max_entries: int = 1024):
	super().__init__()
	self._max_entries = max_entries
	def __getitem__(self, key: tuple[Any, ...]) -> Callable:
	value = super().__getitem__(key)
	self.move_to_end(key)
	return value
	def get(
	self, key: tuple[Any, ...], default: Callable | None = None
	) -> Callable | None:
	if key in self:
	return self[key]
	return default
	def __setitem__(self, key: tuple[Any, ...], value: Callable) -> None:
	if key in self:
	self.move_to_end(key)
	super().__setitem__(key, value)
	if self._max_entries > 0:
	while len(self) > self._max_entries:
	self.popitem(last=False)
	_cache_wrappers: _WrapperRegistry = _WrapperRegistry()
	_runtime_cache_lock = threading.Lock()
	def clear_runtime_cache_wrappers() -> None:
	with _runtime_cache_lock:
	_cache_wrappers.clear()

[Copilot]

The changeset summary line is missing a space after feat:; most tooling that parses conventional commits expects feat: ... (with a space). Updating this improves changelog readability and consistency.

Suggested change

	feat:Allow applying `gr.cache()` to intermediate functions directly
	feat: Allow applying `gr.cache()` to intermediate functions directly