Why ddp is better than ddp_spawn

[binzhougithub]

I am using ddp_spawn strategy now, it seems that ddp is better, because I found this page, see below

========
We use DDP this way because ddp_spawn has a few limitations (due to Python and PyTorch):

1. Since .spawn() trains the model in subprocesses, the model on the main process does not get updated.

2. Dataloader(num_workers=N), where N is large, bottlenecks training with DDP… ie: it will be VERY slow or won’t work at all. This is a PyTorch limitation.

=========

I didn't get these two points, for example, the first argument is about model on main process, my train/validation steps are in subprocesses, therefore, the model on the main process won't bother me. And in 2nd argument, "This is a PyTorch limitation", why doesn't "DDP" strategy have the same issue?

Thanks

[RaoZhongtao]

Same question. Does anyone have any ideas about this?