new perspective to CRNN

[yewalenikhil65]

@jiweiqi
Here is something going on in my mind as i am trying to fit stiff systems.
we know that,
1) X' = CRNN(X(t) ,t) , where CRNN= w_out*(exp.(w_in'* x + w_b))
where w_out, w_in and w_b are parameters.. Naturally, for large systems to be approximated, it is more difficult to train as there are so many parameters in these arrays.

2) X' = w_out * oderatelaws = w_out* [f(X,t) .*k] , where w_out , k are parameters to be learnt (along with function f)
this is another way (although exactly same) to express ODEs of reactions.. Doesn't this mean there are less number of params to be dealt with as compared to first approach ? Is there any benefit by arranging system this way? If yes, is there any way to create a NN out of this ?

[jiweiqi]

Hi @yewalenikhil65 , this is a very brilliant idea. I think this discussion can also be formulated as "is it beneficial to relax part of the physics laws to trade better performance (accuracy, learnability, etc.)". I believe this is worthy and sometimes necessary.

This can be formulated as X' = w_out * [NN(X) .*k], in which we don't use a polynomial form to represent the rate law. I actually did the same way in the yeast oscillation problem in this repo. If you are familiar with Quasi-Steady-State Assumptions, you will find that the reaction rate is usually in rational form, instead of a polynomial form. This is implicitly applied to most biology models. So, to your last question, this is fairly straightforward to implement. By the way, one can augment modern fancy NN architecture of autoencoder, graph neural network into CRNN in such away. We did try to initiate the discussions on how much physics we need in CRNN in the CRNN paper. The flexibility is indeed one of the most important motivations to propose the CRNN framework

The downside is that this will make the learned model depart a little bit from legacy forms. The good news is that the model still preserves quite a lot of interpretability from the information of w_out.

Above are my two cents. We should definitely continue this discussion. If you are excited about it, you can try it in the systems listed in this repo and the biomedical systems you are working on. I will be very happy to collaborate!

[jiweiqi]

Looks interesting.

[yewalenikhil65]

Would you be interested in a google meet someday to discuss more in detail. It's bit straining writing everything

[jiweiqi]

Sure, that's a good idea. Let's do it sometime next week

[yewalenikhil65]

Sure, that's a good idea. Let's do it sometime next week

Let me know time for online meet that will be convenient for you..on email..It would be nice to add my supervisor also formally

[jiweiqi]

https://www.when2meet.com/?11329788-IDATF Let's use this one to schedule a time