Skip to content

Variable value/dtype accessors redo derivations on every weight read #23293

Description

@pctablet505

The torch Variable.value property and the common Variable dtype/autocast accessors redo fixed-cost work on every single weight read. value rebuilds a closure and calls get_device() followed by a no-op str() conversion on each access, and the dtype/autocast path re-checks global autocast state and re-runs standardize_dtype on a dtype that was already normalized at construction time.

This is pure per-call overhead: none of this state can change between reads of an already-built Variable, so redoing it on every forward pass multiplies out across every weight in the model on every call.

Fixed by #23297: hoist the closure to module level, compare value.device.type directly instead of round-tripping through str(), reorder the checks to look at the instance attribute before the global autocast state, and drop the redundant standardize_dtype call.

Part of the per-call Python-dispatch overhead series in #22561.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions