NADE Model Not Learning MNIST Properly

[erdiphd]

I tested the NADE model and trained it for 3500 epochs, but it did not learn the MNIST dataset properly. Below is my output.

I would like to understand the possible reasons for this. Could it be that 3500 epochs is still too low or is there a potential issue with the model itself?

Any insights would be greatly appreciated.

Thank you!

[EugenHotaj]

There may have been some changes with newer versions of pytorch that broke things. I have not run this code in a few years so hard to say off-hand what could be wrong, but 3500 epochs is probably way too many. Have you tried less than that, do the samples look wrong still?

My suggestion would be to look at the training/validation loss curves to see if they look reasonable. If they do, there may be some issues with the sampling code in newer version of pytorch, like some dim= args missing somewhere.

[erdiphd]

Thanks @EugenHotaj for your answer.
I re-trained it again and the loss is discreasing shown as below;

However, the generated images doesnt look like the MNIST dataset.

I will try to investigate the issue and If I find something, I will update here.

[EugenHotaj]

@erdiphd yea if I had to guess there is probably something going wrong with the sampling code.