How to use only fairscale's OSS #13923
Unanswered
toriving
asked this question in
DDP / multi-GPU / multi-node
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Hi,
I want to use only fairscale Optimizer State Sharded (OSS) similar to Deepspeed's zero1, is there any way?
Currently,
ddp_sharded
is in the form of zero2.Strangely, in the case of
ddp_sharded
(zero2), there is an issue that the learning speed is very slow, so I want to use only OSS.Beta Was this translation helpful? Give feedback.
All reactions