When I trained lora model based playground v2.5, it didn't work very well

[plastic0313]

When I trained playground2.5 on my own data set(Contains a series of images and their corresponding tags), I couldn't get good results. But when I use the base model of SD XL, it can be trained very well. What is the reason for this?

[sayakpaul]

The dynamics of EDM training is not fully well-known to us. You should run experiments with different hyperparameters and see what works best. The values provided in the official example lead to reasonable results, though.

[plastic0313]

I have tried a lot of parameter and data annotation methods, but still do not get a usable lora model. The official example is too simple, the puppy is difficult to reflect the reduction degree and generalization ability of the new object, and there may be no way to fine-tune training based on EDM?

[sayakpaul]

DreamBooth is for subject-driven generation and NOT for generalization to new objects. The task is to generate a subject of interest in newer renditions.

[plastic0313]

Yes of course, but I used the same data set, the same training strategy, using SD XL can train very well, successfully generate my own car, but using EDM and playground V2.5, his results are particularly broken, can not get the performance like SD XL Lora on Civital

[sayakpaul]

Training dynamics matter a lot so, I would say that is somewhat expected.