Did anyone get good CIFAR10 results?

[IdoZach]

Hi, thanks for providing this code. I'm trying to reproduce the CIFAR10 results from the original DDPM paper. I use 3x32x32 images, all the CIFAR data (50k frames), 2000 epochs (but I check every 100 epochs how it looks like), and I get some similar results, but not as good as the paper.
This is the result that I get:

I'm also attaching the training results that I get (the divergent one is the validation loss):

My training schedule is similar to the original except that I maximize the batch size on my GPUs. I'm using image size of 32, and U-Net options dim=64, dim_mults=(1,2,4,8).

Was anyone more successful and can share their results and tips? I think that this result is far from perfect. Thanks very much, I hope you could help me find what I'm missing.

[tcapelle]

nop, have been trying all day. MNIST works fine

[yiyixuxu]

you can try p2_loss_weight_gamma = 1. , my result with cifar10 wasn't great either with the default setting but I think you can see a big difference with p2 weighting - the original paper used a reweighted loss too

[we1cao]

How do you show the loss on the tensorboard? Could you share the code maybe?

[malbergo]

does someone have example code for mnist/cifar10?

[tcapelle]

I do here, but with a different codebase: [https://wandb.ai/capecape/train\_sd/reports/How-to-Train-a-Conditional-Diffusion-Model-from-Scratch--VmlldzoyNzIzNTQ1][https_wandb.ai_capecape_train_sd_reports_How-to-Train-a-Conditional-Diffusion-Model-from-Scratch--VmlldzoyNzIzNTQ1] Sent from ProtonMail mobile \-------- Original Message --------

On Nov 5, 2022, 6:30 PM, Michael Albergo < ***@***.***> wrote: does someone have example code for mnist/cifar10? — Reply to this email directly, [view it on GitHub][], or [unsubscribe][]. You are receiving this because you commented.![AEMWOAMOBC4DFILSUZCKI6TWG2KT5A5CNFSM6AAAAAAQZTJ7GOWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTSNYKH4M.gif][]Message ID: ***@***.***>

[https_wandb.ai_capecape_train_sd_reports_How-to-Train-a-Conditional-Diffusion-Model-from-Scratch--VmlldzoyNzIzNTQ1]: https://wandb.ai/capecape/train_sd/reports/How-to-Train-a-Conditional-Diffusion-Model-from-Scratch--VmlldzoyNzIzNTQ1 [view it on GitHub]: #100 (comment) [unsubscribe]: https://github.com/notifications/unsubscribe-auth/AEMWOAIGOGZ2G5PEZKBBUXDWG2KT5ANCNFSM6AAAAAAQZTJ7GM [AEMWOAMOBC4DFILSUZCKI6TWG2KT5A5CNFSM6AAAAAAQZTJ7GOWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTSNYKH4M.gif]: https://github.com/notifications/beacon/AEMWOAMOBC4DFILSUZCKI6TWG2KT5A5CNFSM6AAAAAAQZTJ7GOWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTSNYKH4M.gif

[Gregory1994]

similar problem, any solution please?

[pimakshay]

What parameter is 'p2_loss_weight_gamma'? and where do we exactly use it? Can anyone highlight it in the denoising_diffusion_pytorch.py file?

[chengyiqiu1121]

you can try p2_loss_weight_gamma = 1. , my result with cifar10 wasn't great either with the default setting but I think you can see a big difference with p2 weighting - the original paper used a reweighted loss too

Can u tell me where is p2_loss_weight_gamma, i did not find it