Error in densenet lr update formula:

[nivdu]

The 0.05 in the update LR formula overrides the initialized LR per layer (the 1e-1).
The line that inits the LR per layer with the scale_lr option is overridden by the formula.
m.lr = 1e-1 / m.lr_ratio if self.scale_lr else 1e-1

The formula with the change:
self.optim.param_groups[i]['lr'] = (m.lr)*(1+np.cos(np.pi*self.j/m.max_j))