Training a Multi-Speaker Model from a Single-Speaker Checkpoint

[lpscr]

Before I begin, I want to say a big thank you for all your amazing work!

I'm currently using a checkpoint to successfully fine-tune a single speaker's voice. However, I encounter an issue when attempting to do the same for multiple speakers. It leads to the training process start for begin and the loss of the checkpoint I'm using.

I'm wondering if it's possible to employ a checkpoint model for fine-tuning with multiple speakers. Currently, I'm forced to fine-tune each speaker individually, which consumes a significant amount of time and storage space and a lot files models. It would be far more convenient to have a single model that can accommodate the speakers I need.

Thank you ,

[egorsmkv]

I think you can initialize a model with few speakers and then just transfer weights to this model from the single speaker model.

[lpscr]

thank you p0p4k like you write the egorsmkv , is this possible?

[p0p4k]

Did you try training with multiple speakers first?

[lpscr]

right now i train a single speaker i use the pretrained checkpoints in front page you have already thank you for share
i copy the G,D rename G_0,D_O and then start the train like this get the pretrained checkpoints
with just some steps of train i get good model sound ,

i want do the same with multi speaker and keep the pretrained checkpoints
when train start it's reset all the check point when it's see it's multispeaker

yes i can train a multi speaker from the begin with out use the pretrained checkpoints , this working fine
but very bad quality with noise sound and a lot of hour train

now this i do need run the train 10 times to get speaker good quality
so need to do one by one with pretrained checkpoints
and i have 10GB size for 10 models ,

the nice it's the method with single speakers working also with multispeakers
like this only one time train, 1GB size of model , and i can easy call with id 0,1,..

is this possible ?

if no
also there is way to covert the 10 models files to 1 mode file and call with id
like multispeaker model?

and last

it's possible when i have a multi speaker model to add more extend the model
so let say i have done train with multi speaker model 10 speaker
and i need to add one more , to call with id 11 , so like this i can add more speaker
and i have only one file model

thank you

[p0p4k]

Ok, I can give you a reference code https://github.com/anonymous-pits/pits/blob/main/utils.py (check load_checkpoint_diffsize and load_checkpoint functions.) Try to understand it and make it work in vits2. You will get what you need.

[lpscr]

Thank you for seeing it ,I will try the reference code then,