Galaxy format

[soulios]

• Saving now only weights and not whole models though callbacks with N epochs frequency.
• Fixed issue with the history plots.

[bernt-matthias]

Can you fix the error: dfpl: error: the following arguments are required: --test_path, --preds_path?

[soulios]

Can you fix the error: dfpl: error: the following arguments are required: --test_path, --preds_path?

Fixed it. ready for review. @mai00fti @bernt-matthias

[tom-mohr]

If epochs < 10, the output looks like this:

.
├── autoencoder
│   ├── autoencoder_weights.h5.history.csv
│   ├── autoencoder_weights.h5.history.svg
│   └── encoder_weights.h5
├── conda_activate.log
├── config.json
└── model
    ├── AR
    │   ├── history.csv
    │   └── history.svg
    └── train.log

If epochs >= 10, the output looks like this:

.
├── autoencoder
│   ├── autoencoder_weights.h5.history.csv
│   ├── autoencoder_weights.h5.history.svg
│   └── encoder_weights.h5
├── conda_activate.log
├── config.json
└── model
    ├── AR
    │   ├── auc_data.png
    │   ├── history.csv
    │   ├── history.svg
    │   ├── model_weights.hdf5
    │   ├── predicted.testdata.aucdata.csv
    │   ├── predicted.testdata.csv
    │   └── predicted.testdata.prec_rec_f1.csv
    ├── Aromatase
    │   ├── auc_data.png
    │   ├── history.csv
    │   ├── history.svg
    │   ├── model_weights.hdf5
    │   ├── predicted.testdata.aucdata.csv
    │   ├── predicted.testdata.csv
    │   └── predicted.testdata.prec_rec_f1.csv
    ├── ED
    │   ├── auc_data.png
    │   ├── history.csv
    │   ├── history.svg
    │   ├── model_weights.hdf5
    │   ├── predicted.testdata.aucdata.csv
    │   ├── predicted.testdata.csv
    │   └── predicted.testdata.prec_rec_f1.csv
    ├── ER
    │   ├── auc_data.png
    │   ├── history.csv
    │   ├── history.svg
    │   ├── model_weights.hdf5
    │   ├── predicted.testdata.aucdata.csv
    │   ├── predicted.testdata.csv
    │   └── predicted.testdata.prec_rec_f1.csv
    ├── GR
    │   ├── auc_data.png
    │   ├── history.csv
    │   ├── history.svg
    │   ├── model_weights.hdf5
    │   ├── predicted.testdata.aucdata.csv
    │   ├── predicted.testdata.csv
    │   └── predicted.testdata.prec_rec_f1.csv
    ├── PPARg
    │   ├── auc_data.png
    │   ├── history.csv
    │   ├── history.svg
    │   ├── model_weights.hdf5
    │   ├── predicted.testdata.aucdata.csv
    │   ├── predicted.testdata.csv
    │   └── predicted.testdata.prec_rec_f1.csv
    ├── single_label_random_model.evaluation.csv
    ├── TR
    │   ├── auc_data.png
    │   ├── history.csv
    │   ├── history.svg
    │   ├── model_weights.hdf5
    │   ├── predicted.testdata.aucdata.csv
    │   ├── predicted.testdata.csv
    │   └── predicted.testdata.prec_rec_f1.csv
    └── train.log

This is because in the first case, an error is thrown when the first h5 file is created:

FileNotFoundError: [Errno 2] Unable to synchronously open file (unable to open file: name = '/tmp/tmpvbmwtqa8/job_working_directory/000/4/working/model/AR/model_weights.hdf5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)

This originates from single_label_model.py:555:

    callback_model.load_weights(filepath=checkpoint_model_weights_path)

[soulios]

If epochs < 10, the output looks like this:

.
├── autoencoder
│   ├── autoencoder_weights.h5.history.csv
│   ├── autoencoder_weights.h5.history.svg
│   └── encoder_weights.h5
├── conda_activate.log
├── config.json
└── model
    ├── AR
    │   ├── history.csv
    │   └── history.svg
    └── train.log

If epochs >= 10, the output looks like this:

.
├── autoencoder
│   ├── autoencoder_weights.h5.history.csv
│   ├── autoencoder_weights.h5.history.svg
│   └── encoder_weights.h5
├── conda_activate.log
├── config.json
└── model
    ├── AR
    │   ├── auc_data.png
    │   ├── history.csv
    │   ├── history.svg
    │   ├── model_weights.hdf5
    │   ├── predicted.testdata.aucdata.csv
    │   ├── predicted.testdata.csv
    │   └── predicted.testdata.prec_rec_f1.csv
    ├── Aromatase
    │   ├── auc_data.png
    │   ├── history.csv
    │   ├── history.svg
    │   ├── model_weights.hdf5
    │   ├── predicted.testdata.aucdata.csv
    │   ├── predicted.testdata.csv
    │   └── predicted.testdata.prec_rec_f1.csv
    ├── ED
    │   ├── auc_data.png
    │   ├── history.csv
    │   ├── history.svg
    │   ├── model_weights.hdf5
    │   ├── predicted.testdata.aucdata.csv
    │   ├── predicted.testdata.csv
    │   └── predicted.testdata.prec_rec_f1.csv
    ├── ER
    │   ├── auc_data.png
    │   ├── history.csv
    │   ├── history.svg
    │   ├── model_weights.hdf5
    │   ├── predicted.testdata.aucdata.csv
    │   ├── predicted.testdata.csv
    │   └── predicted.testdata.prec_rec_f1.csv
    ├── GR
    │   ├── auc_data.png
    │   ├── history.csv
    │   ├── history.svg
    │   ├── model_weights.hdf5
    │   ├── predicted.testdata.aucdata.csv
    │   ├── predicted.testdata.csv
    │   └── predicted.testdata.prec_rec_f1.csv
    ├── PPARg
    │   ├── auc_data.png
    │   ├── history.csv
    │   ├── history.svg
    │   ├── model_weights.hdf5
    │   ├── predicted.testdata.aucdata.csv
    │   ├── predicted.testdata.csv
    │   └── predicted.testdata.prec_rec_f1.csv
    ├── single_label_random_model.evaluation.csv
    ├── TR
    │   ├── auc_data.png
    │   ├── history.csv
    │   ├── history.svg
    │   ├── model_weights.hdf5
    │   ├── predicted.testdata.aucdata.csv
    │   ├── predicted.testdata.csv
    │   └── predicted.testdata.prec_rec_f1.csv
    └── train.log

This is because in the first case, an error is thrown when the first h5 file is created:

FileNotFoundError: [Errno 2] Unable to synchronously open file (unable to open file: name = '/tmp/tmpvbmwtqa8/job_working_directory/000/4/working/model/AR/model_weights.hdf5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)

This originates from single_label_model.py:555:

    callback_model.load_weights(filepath=checkpoint_model_weights_path)

Yes, because the period in which the model is saved is every 10 epochs because if it was every epoch it would add a lot of time overhead. This is intended. But maybe we should limit the opts.epochs to take values above 10 and the same goes for autoencoder but it is saved every 5 epochs, so opts.aeEpochs should accept minimum 5 as a value.

[bernt-matthias]

Yes, because the period in which the model is saved is every 10 epochs because if it was every epoch it would add a lot of time overhead. This is intended. But maybe we should limit the opts.epochs to take values above 10 and the same goes for autoencoder but it is saved every 5 epochs, so opts.aeEpochs should accept minimum 5 as a value.

Sounds reasonable.

@tom-mohr has already set the min for epochs in the Galaxy tool.. then we do the same for aeEpochs. Should we also do this in the python code, e.g. via argparse .. but then the json input is again not covered?
Edit: Should the values be a multiple of 10 (resp. 5)?

Just thinking loud (and not suggesting that we should do this in the PR): We can also give the list of CLI arguments to parser.parse_args (e.g. parser.parse_args(['--foo', 'FOO'])). Could we generate such a list from the json content. Then all argument parsing would be via argparse....