The torch Variable.value property and the common Variable dtype/autocast accessors redo fixed-cost work on every single weight read. value rebuilds a closure and calls get_device() followed by a no-op str() conversion on each access, and the dtype/autocast path re-checks global autocast state and re-runs standardize_dtype on a dtype that was already normalized at construction time.
This is pure per-call overhead: none of this state can change between reads of an already-built Variable, so redoing it on every forward pass multiplies out across every weight in the model on every call.
Fixed by #23297: hoist the closure to module level, compare value.device.type directly instead of round-tripping through str(), reorder the checks to look at the instance attribute before the global autocast state, and drop the redundant standardize_dtype call.
Part of the per-call Python-dispatch overhead series in #22561.
The torch
Variable.valueproperty and the commonVariabledtype/autocast accessors redo fixed-cost work on every single weight read.valuerebuilds a closure and callsget_device()followed by a no-opstr()conversion on each access, and the dtype/autocast path re-checks global autocast state and re-runsstandardize_dtypeon a dtype that was already normalized at construction time.This is pure per-call overhead: none of this state can change between reads of an already-built
Variable, so redoing it on every forward pass multiplies out across every weight in the model on every call.Fixed by #23297: hoist the closure to module level, compare
value.device.typedirectly instead of round-tripping throughstr(), reorder the checks to look at the instance attribute before the global autocast state, and drop the redundantstandardize_dtypecall.Part of the per-call Python-dispatch overhead series in #22561.