DDP, find_unused_parameters and manual optimisation #8000
Unanswered
DinoMan
asked this question in
DDP / multi-GPU / multi-node
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I am running my model on multiple GPUs and because it is a GAN I use manual optimisation. I initially got a warning to set find_unused_parameters to False since no unused params were found. I did this but then got another warning saying that manual optimisation does not support this. Can someone please explain why this happens. What the hit of setting find_unused_parameters to True is and if there is a solution (or workaround) so that I can set find_unused_parameters to False and continue to use manual optimisation.
Beta Was this translation helpful? Give feedback.
All reactions