Exp16: Evaluate NVP_6

[Micky774]

Using the same hyper-parameters as in experiment 15 (#40), evaluate NVP_6 in comparison to NVP_5 and NVP_4

[Micky774]

Scripted and ready to run.

[Micky774]

Extending NVP_6 run to 120 epochs to achieve comparable training time to NVP_4

[Micky774]

Running

[Micky774]

Results show NVP_6 being competitive with NVP_4 by number of weight updates, however NVP_6 performs more weight updates per unit time so it outperforms NVP_4 with respect to time.

However, note that in the extended trial both models run for similar times however NVP_6 does not beat out NVP_4 as was predicted -- it instead reaches a lower bound on loss that it fails to surpass. This may be alleviated by learning rate / batch size scheduling, and should be revisited later.

We should also consider extending NVP_4's run to match the extended NVP_6 run in terms of weight update count to determine whether NVP_4 suffers from the same or similar lower bound on loss convergence.

[Micky774]

Extending NVP_4 to run for an additional 30 epochs, reaching 120 epochs total to match NVP_6 to determine whether one has a lower floor of convergence for loss than the other with all else being equal.