How to implement a local loss supervison?

[2catycm]

Some model like Mixture of Experts, would add some local loss at specific layers, which is independent w.r.t. the training loss for the whole network.