pretrained model example

[9B8DY6]

The pretrained model is fit into UK biobank of which age distribution is [42,64]. But my dataset age distribution is [20,86].
If I use it for validation, would it be not working well on my dataset?
I have tried to fine-tune pretrained model, but your pretrained model outputs_dim=40, mine 70...it does not match...
What can I do?

[ha-ha-ha-han]

Good point! I am developing a tutorial on this transfer learning issue. The basic idea is to replace the final layer with your targeted task, and then only retrain the last layer with a large learning rate. Then you can finetune the full model.
Feel free to star/watch the repository so that you'll be updated when I post the tutorial.

[9B8DY6]

Good point! I am developing a tutorial on this transfer learning issue. The basic idea is to replace the final layer with your targeted task, and then only retrain the last layer with a large learning rate. Then you can finetune the full model.
Feel free to star/watch the repository so that you'll be updated when I post the tutorial.

I tried to do transfer learning but when I changed output dims as 70, it did not work with RuntimeError even if with 'strict=False'
Could you tell me what is wrong?

[yilei-wu]

https://pytorch.org/tutorials/beginner/saving_loading_models.html
'strict=False' will ignore mismatch for the loading when key is different. If you only modify the shape of the output layer, the key name woundn't change automatically.
If you prefer not to change the code, the alternative is load a pat of the state_dict. you may refer (https://discuss.pytorch.org/t/how-to-load-part-of-pre-trained-model/1113)

[abanic7]

Good point! I am developing a tutorial on this transfer learning issue. The basic idea is to replace the final layer with your targeted task, and then only retrain the last layer with a large learning rate. Then you can finetune the full model. Feel free to star/watch the repository so that you'll be updated when I post the tutorial.

When are you going to publish this? I can't seem to replicate your results, i'm getting MAE of around 8 for IXI database.